mirror of
https://github.com/pcvolkmer/etl-processor.git
synced 2025-04-19 17:26:51 +00:00
Compare commits
135 Commits
Author | SHA1 | Date | |
---|---|---|---|
c6b37fda69 | |||
8e3de6a220 | |||
c5c553f817 | |||
7d97365aea | |||
48b1e62e22 | |||
66cc818755 | |||
9d4786fae3 | |||
b78dc3519b | |||
46015c5b66 | |||
a4d0b73d2b | |||
9307fc0dad | |||
586d388e57 | |||
7ae34719fd | |||
033750eb10 | |||
befeef3153 | |||
98b971d7db | |||
56a63b276e | |||
c0ea5fcd51 | |||
d4fd54f51f | |||
d49671f0d4 | |||
84868dc22c | |||
4ad6c4bd0a | |||
9bdd8ba375 | |||
f027339425 | |||
3c5639708f | |||
639159c677 | |||
38261d6d2c | |||
47ebe46974 | |||
f347653be8 | |||
775a7df1ce | |||
f66b737f11 | |||
3a19212a78 | |||
280fbd445e | |||
91e2cf5ef1 | |||
262c54f2e5 | |||
|
b25e580113 | ||
ff27b7157d | |||
1e652a7856 | |||
74ff9f08a4 | |||
23cc2f365a | |||
53b4cf1a95 | |||
5ce13e962b | |||
3257493b6a | |||
2036077c06 | |||
8ce3aed870 | |||
998989d319 | |||
e95fa2fb12 | |||
1bcc8c13de | |||
2fc3299543 | |||
5575867632 | |||
46ba565c29 | |||
|
6cdbd35e64 | ||
|
d258d9081b | ||
eb49ba611b | |||
efa736f232 | |||
4a7030e85b | |||
464c8b8c1d | |||
3f1bb4f4e2 | |||
370ea87095 | |||
c8f6e6efc8 | |||
c949ec07e5 | |||
6e0ec6b95a | |||
0ff56416dd | |||
3a2f6a2bb6 | |||
3eb9e68786 | |||
59403d1dba | |||
9f5ac664af | |||
5867ed9dd3 | |||
4d6d1879e6 | |||
2a34c0efc9 | |||
0ee00de5aa | |||
baeebdb9b8 | |||
5320466b6c | |||
263cb02416 | |||
0b37fd7091 | |||
bdee969409 | |||
4c39920afd | |||
5e836c48b0 | |||
fb5a3c062c | |||
8fc0609aa4 | |||
a846a8765a | |||
8645becd82 | |||
011511d5ef | |||
e9839c2731 | |||
86bee9e2cf | |||
f419acb924 | |||
52171e8ebe | |||
a2124ba83d | |||
a046203339 | |||
b40d41ce8c | |||
57de96771c | |||
3bc148f7ea | |||
8e6b1ec799 | |||
8e5f5c73ec | |||
d4f984b138 | |||
24ebbf3b50 | |||
9c6bd64a7e | |||
6567aa803c | |||
e874350712 | |||
94d7b4c4f0 | |||
107429fda7 | |||
26b2f65e67 | |||
e863269a42 | |||
4ab95ef11f | |||
2244ef1b86 | |||
c3ddb387e2 | |||
ae5d8341cc | |||
40b2558943 | |||
9a6a0c6138 | |||
5985327219 | |||
06f9e8ace9 | |||
365a651918 | |||
5fcc24f915 | |||
3bd7239812 | |||
1672ad8640 | |||
710aeb1f18 | |||
06d11790b6 | |||
959f6889d4 | |||
0f5a68660d | |||
b809a2da02 | |||
effffcfc1a | |||
7b3151d227 | |||
26b415f336 | |||
bda3c30a74 | |||
8779600330 | |||
159fb46009 | |||
eabbbfbb68 | |||
4db38ef2f0 | |||
ed6d21e920 | |||
550bee5ad3 | |||
bd7dccbd87 | |||
8ae958b8c4 | |||
0f144568e3 | |||
08540e3bd7 | |||
43af1aa103 |
6
.github/workflows/deploy.yml
vendored
6
.github/workflows/deploy.yml
vendored
@ -30,6 +30,6 @@ jobs:
|
||||
- name: Execute image build and push
|
||||
run: |
|
||||
./gradlew bootBuildImage
|
||||
docker tag ghcr.io/ccc-mf/etl-processor ghcr.io/ccc-mf/etl-processor:${{ github.ref_name }}
|
||||
docker push ghcr.io/ccc-mf/etl-processor
|
||||
docker push ghcr.io/ccc-mf/etl-processor:${{ github.ref_name }}
|
||||
docker tag ghcr.io/${{ github.repository }} ghcr.io/${{ github.repository }}:${{ github.ref_name }}
|
||||
docker push ghcr.io/${{ github.repository }}
|
||||
docker push ghcr.io/${{ github.repository }}:${{ github.ref_name }}
|
91
README.md
91
README.md
@ -1,6 +1,6 @@
|
||||
# ETL-Processor for bwHC data [](https://github.com/CCC-MF/etl-processor/actions/workflows/test.yml)
|
||||
# ETL-Processor for DNPM:DIP [](https://github.com/pcvolkmer/etl-processor/actions/workflows/test.yml)
|
||||
|
||||
Diese Anwendung versendet ein bwHC-MTB-File an das bwHC-Backend und pseudonymisiert die Patienten-ID.
|
||||
Diese Anwendung versendet ein bwHC-MTB-File im bwHC-Datenmodell 1.0 an DNPM:DIP und pseudonymisiert die Patienten-ID.
|
||||
|
||||
## Einordnung innerhalb einer DNPM-ETL-Strecke
|
||||
|
||||
@ -9,7 +9,7 @@ Diese Anwendung erlaubt das Entgegennehmen von HTTP/REST-Anfragen aus dem Onkost
|
||||
Der Inhalt einer Anfrage, wenn ein bwHC-MTBFile, wird pseudonymisiert und auf Duplikate geprüft.
|
||||
Duplikate werden verworfen, Änderungen werden weitergeleitet.
|
||||
|
||||
Löschanfragen werden immer als Löschanfrage an das bwHC-backend weitergeleitet.
|
||||
Löschanfragen werden immer als Löschanfrage an DNPM:DIP weitergeleitet.
|
||||
|
||||
Zudem ist eine minimalistische Weboberfläche integriert, die einen Einblick in den aktuellen Zustand der Anwendung gewährt.
|
||||
|
||||
@ -22,7 +22,17 @@ Die Erkennung von Duplikaten ist normalerweise immer aktiv, kann jedoch über de
|
||||
|
||||
### Datenübermittlung über HTTP/REST
|
||||
|
||||
Anfragen werden, wenn nicht als Duplikat behandelt, nach der Pseudonymisierung direkt an das bwHC-Backend gesendet.
|
||||
Anfragen werden, wenn nicht als Duplikat behandelt, nach der Pseudonymisierung direkt an DNPM:DIP gesendet.
|
||||
|
||||
Ein HTTP Request kann, angenommen die Installation erfolgte auf dem Host `dnpm.example.com` an nachfolgende URLs gesendet werden:
|
||||
|
||||
| HTTP-Request | URL | Consent-Status im Datensatz | Bemerkung |
|
||||
|--------------|-----------------------------------------|-----------------------------|---------------------------------------------------------------------------------|
|
||||
| `POST` | `https://dnpm.example.com/mtb` | `ACTIVE` | Die Anwendung verarbeitet den eingehenden Datensatz |
|
||||
| `POST` | `https://dnpm.example.com/mtb` | `REJECT` | Die Anwendung sendet einen Lösch-Request für die im Datensatz angegebene Pat-ID |
|
||||
| `DELETE` | `https://dnpm.example.com/mtb/12345678` | - | Die Anwendung sendet einen Lösch-Request für Pat-ID `12345678` |
|
||||
|
||||
Anstelle des Pfads `/mtb` kann auch, wie in Version 0.9 und älter üblich, `/mtbfile` verwendet werden.
|
||||
|
||||
### Datenübermittlung mit Apache Kafka
|
||||
|
||||
@ -33,6 +43,21 @@ Siehe hierzu auch: https://github.com/CCC-MF/kafka-to-bwhc
|
||||
|
||||
## Konfiguration
|
||||
|
||||
### 🔥 Wichtige Änderungen in Version 0.10
|
||||
|
||||
Ab Version 0.10 wird [DNPM:DIP](https://github.com/dnpm-dip) unterstützt und als Standardendpunkt verwendet.
|
||||
Soll noch das alte bwHC-Backend verwendet werden, so ist die Umgebungsvariable `APP_REST_IS_BWHC` auf `true` zu setzen.
|
||||
|
||||
### 🔥 Breaking Changes nach Version 0.10
|
||||
|
||||
In Versionen des ETL-Processors **nach Version 0.10** werden die folgenden Konfigurationsoptionen entfernt:
|
||||
|
||||
* `APP_KAFKA_TOPIC`: Nutzen Sie nun die Konfigurationsoption `APP_KAFKA_OUTPUT_TOPIC`
|
||||
* `APP_KAFKA_RESPONSE_TOPIC`: Nutzen Sie nun die Konfigurationsoption `APP_KAFKA_OUTPUT_RESPONSE_TOPIC`
|
||||
|
||||
Der Pfad zum Versenden von MTB-Daten ist nun offiziell `/mtb`.
|
||||
In Versionen **nach Version 0.10** wird die Unterstützung des Pfads `/mtbfile` entfernt.
|
||||
|
||||
### Pseudonymisierung der Patienten-ID
|
||||
|
||||
Wenn eine URI zu einer gPAS-Instanz (Version >= 2023.1.0) angegeben ist, wird diese verwendet.
|
||||
@ -41,11 +66,9 @@ Ist diese nicht gesetzt. wird intern eine Anonymisierung der Patienten-ID vorgen
|
||||
* `APP_PSEUDONYMIZE_PREFIX`: Standortbezogenes Präfix - `UNKNOWN`, wenn nicht gesetzt
|
||||
* `APP_PSEUDONYMIZE_GENERATOR`: `BUILDIN` oder `GPAS` - `BUILDIN`, wenn nicht gesetzt
|
||||
|
||||
**Hinweise**:
|
||||
**Hinweis**
|
||||
|
||||
* Der alte Konfigurationsparameter `APP_PSEUDONYMIZER` mit den Werten `GPAS` oder `BUILDIN` sollte nicht mehr verwendet
|
||||
werden.
|
||||
* Die Pseudonymisierung erfolgt im ETL-Prozessor nur für die Patienten-ID.
|
||||
Die Pseudonymisierung erfolgt im ETL-Prozessor nur für die Patienten-ID.
|
||||
Andere IDs werden mithilfe des standortbezogenen Präfixes (erneut) anonymisiert, um für den aktuellen Kontext nicht
|
||||
vergleichbare IDs bereitzustellen.
|
||||
|
||||
@ -63,10 +86,6 @@ Wurde die Verwendung von gPAS konfiguriert, so sind weitere Angaben zu konfiguri
|
||||
* `APP_PSEUDONYMIZE_GPAS_TARGET`: gPas Domänenname
|
||||
* `APP_PSEUDONYMIZE_GPAS_USERNAME`: gPas Basic-Auth Benutzername
|
||||
* `APP_PSEUDONYMIZE_GPAS_PASSWORD`: gPas Basic-Auth Passwort
|
||||
* ~~`APP_PSEUDONYMIZE_GPAS_SSLCALOCATION`~~: **Veraltet** - Root Zertifikat für gPas, falls es dediziert hinzugefügt werden muss.
|
||||
|
||||
Der Konfigurationsparameter `APP_PSEUDONYMIZE_GPAS_SSLCALOCATION` sollte nicht mehr verwendet werden und wird in einer kommenden Version entfernt.
|
||||
Stattdessen sollte das Root Zertifikat wie unter [_Integration eines eigenen Root CA Zertifikats_](#integration-eines-eigenen-root-ca-zertifikats) beschrieben eingebunden werden.
|
||||
|
||||
### Anmeldung mit einem Passwort
|
||||
|
||||
@ -133,7 +152,7 @@ Sie bekommen dabei wieder die Standardrolle zugewiesen.
|
||||
#### Auswirkungen auf den dargestellten Inhalt
|
||||
|
||||
Nur Administratoren haben Zugriff auf den Konfigurationsbereich, nur angemeldete Benutzer können die anonymisierte oder
|
||||
pseudonymisierte Patienten-ID sowie den Qualitätsbericht des bwHC-Backends einsehen.
|
||||
pseudonymisierte Patienten-ID sowie den Qualitätsbericht von DNPM:DIP einsehen.
|
||||
|
||||
Wurde kein Administrator-Account konfiguriert, sind diese Inhalte generell nicht verfügbar.
|
||||
|
||||
@ -149,7 +168,7 @@ zur Nutzung des MTB-File-Endpunkts eine HTTP-Basic-Authentifizierung voraussetze
|
||||
|
||||

|
||||
|
||||
In diesem Fall können den Endpunkt für das Onkostar-Plugin **[onkostar-plugin-dnpmexport](https://github.com/CCC-MF/onkostar-plugin-dnpmexport)** wie folgt konfigurieren:
|
||||
In diesem Fall kann der Endpunkt für das Onkostar-Plugin **[onkostar-plugin-dnpmexport](https://github.com/CCC-MF/onkostar-plugin-dnpmexport)** wie folgt konfiguriert werden:
|
||||
|
||||
```
|
||||
https://testonkostar:MTg1NTL...NGU4@etl.example.com/mtbfile
|
||||
@ -157,10 +176,12 @@ https://testonkostar:MTg1NTL...NGU4@etl.example.com/mtbfile
|
||||
|
||||
Ist die Verwendung von Tokens aktiv, werden Anfragen ohne die Angabe der Token-Information abgelehnt.
|
||||
|
||||
Alternativ kann eine Authentifizierung über Benutzername/Passwort oder OIDC erfolgen.
|
||||
|
||||
### Transformation von Werten
|
||||
|
||||
In Onkostar kann es vorkommen, dass ein Wert eines Merkmalskatalogs an einem Standort angepasst wurde und dadurch nicht dem Wert entspricht,
|
||||
der vom bwHC-Backend akzeptiert wird.
|
||||
der von DNPM:DIP akzeptiert wird.
|
||||
|
||||
Diese Anwendung bietet daher die Möglichkeit, eine Transformation vorzunehmen. Hierzu muss der "Pfad" innerhalb des JSON-MTB-Files angegeben werden und
|
||||
welcher Wert wie ersetzt werden soll.
|
||||
@ -180,18 +201,21 @@ Werden sowohl REST als auch Kafka-Endpunkt konfiguriert, wird nur der REST-Endpu
|
||||
|
||||
#### REST
|
||||
|
||||
Folgende Umgebungsvariablen müssen gesetzt sein, damit ein bwHC-MTB-File an das bwHC-Backend gesendet wird:
|
||||
Folgende Umgebungsvariablen müssen gesetzt sein, damit ein bwHC-MTB-File an DNPM:DIP gesendet wird:
|
||||
|
||||
* `APP_REST_URI`: URI der zu benutzenden API der bwHC-Backend-Instanz. z.B.: `http://localhost:9000/bwhc/etl/api`
|
||||
* `APP_REST_URI`: URI der zu benutzenden API der Backend-Instanz. Zum Beispiel:
|
||||
* `http://localhost:9000/bwhc/etl/api` für **bwHC Backend**
|
||||
* `http://localhost:9000/api` für **dnpm:dip**
|
||||
* `APP_REST_USERNAME`: Basic-Auth-Benutzername für den REST-Endpunkt
|
||||
* `APP_REST_PASSWORD`: Basic-Auth-Passwort für den REST-Endpunkt
|
||||
* `APP_REST_IS_BWHC`: `true` für **bwHC Backend**, weglassen oder `false` für **dnpm:dip**
|
||||
|
||||
#### Kafka-Topics
|
||||
|
||||
Folgende Umgebungsvariablen müssen gesetzt sein, damit ein bwHC-MTB-File an ein Kafka-Topic übermittelt wird:
|
||||
|
||||
* `APP_KAFKA_OUTPUT_TOPIC`: Zu verwendendes Topic zum Versenden von Anfragen.
|
||||
Ersetzt in einer kommenden Version `APP_KAFKA_TOPIC`.
|
||||
* `APP_KAFKA_OUTPUT_RESPONSE_TOPIC`: Topic mit Antworten über den Erfolg des Versendens. Standardwert: `APP_KAFKA_TOPIC` mit Anhang "_response".
|
||||
Ersetzt in einer kommenden Version `APP_KAFKA_RESPONSE_TOPIC`.
|
||||
* `APP_KAFKA_GROUP_ID`: Kafka GroupID des Consumers. Standardwert: `APP_KAFKA_TOPIC` mit Anhang "_group".
|
||||
* `APP_KAFKA_SERVERS`: Zu verwendende Kafka-Bootstrap-Server als kommagetrennte Liste
|
||||
|
||||
@ -199,7 +223,7 @@ Wird keine Rückantwort über Apache Kafka empfangen und es gibt keine weitere M
|
||||
|
||||
Weitere Einstellungen können über die Parameter von Spring Kafka konfiguriert werden.
|
||||
|
||||
Lässt sich keine Verbindung zu dem bwHC-Backend aufbauen, wird eine Rückantwort mit Status-Code `900` erwartet, welchen es
|
||||
Lässt sich keine Verbindung zu dem Backend aufbauen, wird eine Rückantwort mit Status-Code `900` erwartet, welchen es
|
||||
für HTTP nicht gibt.
|
||||
|
||||
Wird die Umgebungsvariable `APP_KAFKA_INPUT_TOPIC` gesetzt, kann eine Nachricht auch über dieses Kafka-Topic an den ETL-Prozessor übermittelt werden.
|
||||
@ -236,7 +260,7 @@ kafka-configs.sh --bootstrap-server localhost:9092 --alter --topic test --add-co
|
||||
Da als Key eines Records die (pseudonymisierte) Patienten-ID verwendet wird, stehen mit obiger Konfiguration
|
||||
der Kafka-Topics nach 10 Sekunden nur noch der jeweils letzte Eintrag für den entsprechenden Key zur Verfügung.
|
||||
|
||||
Da der Key sowohl für die Records in Richtung bwHC-Backend für die Rückantwort identisch aufgebaut ist, lassen sich so
|
||||
Da der Key sowohl für die Records in Richtung DNPM:DIP, als auch für die Rückantwort identisch aufgebaut ist, lassen sich so
|
||||
auch im Falle eines Consent-Widerspruchs die enthaltenen Daten als auch die Offenlegung durch Verifikationsdaten in der
|
||||
Antwort effektiv verhindern, da diese nach 10 Sekunden gelöscht werden.
|
||||
|
||||
@ -245,9 +269,30 @@ ein Consent-Widerspruch erfolgte.
|
||||
|
||||
Dieses Vorgehen empfiehlt sich, wenn Sie gespeicherte Records nachgelagert für andere Auswertungen verwenden möchten.
|
||||
|
||||
### Antworten und Statusauswertung
|
||||
|
||||
Anfragen an das bwHC-Backend aus Versionen bis 0.9.x wurden wie folgt behandelt:
|
||||
|
||||
| HTTP-Response | Status |
|
||||
|----------------|-----------|
|
||||
| `HTTP 200` | `SUCCESS` |
|
||||
| `HTTP 201` | `WARNING` |
|
||||
| `HTTP 400-...` | `ERROR` |
|
||||
|
||||
Dies konnte dazu führen, dass zwar mit einem `HTTP 201` geantwortet wurde, aber dennoch in der Issue-Liste die
|
||||
Severity `error` aufgetaucht ist.
|
||||
|
||||
Ab Version 0.10 wird die Issue-Liste der Antwort verwendet und die darion enthaltene höchste Severity-Stufe als Ergebnis verwendet.
|
||||
|
||||
| Höchste Severity | Status |
|
||||
|------------------|-----------|
|
||||
| `info` | `SUCCESS` |
|
||||
| `warning` | `WARNING` |
|
||||
| `error`, `fatal` | `ERROR` |
|
||||
|
||||
## Docker-Images
|
||||
|
||||
Diese Anwendung ist auch als Docker-Image verfügbar: https://github.com/CCC-MF/etl-processor/pkgs/container/etl-processor
|
||||
Diese Anwendung ist auch als Docker-Image verfügbar: https://github.com/pcvolkmer/etl-processor/pkgs/container/etl-processor
|
||||
|
||||
### Images lokal bauen
|
||||
|
||||
@ -360,3 +405,5 @@ Die Datei `application-dev.yml` enthält hierzu die Konfiguration für das Profi
|
||||
|
||||
Beim Ausführen der Integrationstests wird eine Testdatenbank in einem Docker-Container gestartet.
|
||||
Siehe hier auch die Klasse `AbstractTestcontainerTest` unter `src/integrationTest`.
|
||||
|
||||
Ein einfaches Entwickler-Setup inklusive DNPM:DIP ist mit Hilfe von https://github.com/pcvolkmer/dnpmdip-devenv realisierbar.
|
@ -1,30 +1,36 @@
|
||||
import org.gradle.api.tasks.testing.logging.TestLogEvent
|
||||
import org.jetbrains.kotlin.gradle.dsl.JvmTarget
|
||||
import org.jetbrains.kotlin.gradle.tasks.KotlinCompile
|
||||
import org.springframework.boot.gradle.tasks.bundling.BootBuildImage
|
||||
|
||||
plugins {
|
||||
war
|
||||
id("org.springframework.boot") version "3.2.7"
|
||||
id("io.spring.dependency-management") version "1.1.5"
|
||||
kotlin("jvm") version "1.9.24"
|
||||
kotlin("plugin.spring") version "1.9.24"
|
||||
id("org.springframework.boot") version "3.4.4"
|
||||
id("io.spring.dependency-management") version "1.1.7"
|
||||
kotlin("jvm") version "1.9.25"
|
||||
kotlin("plugin.spring") version "1.9.25"
|
||||
jacoco
|
||||
}
|
||||
|
||||
group = "de.ukw.ccc"
|
||||
version = "0.9.6"
|
||||
group = "dev.dnpm"
|
||||
version = "0.11.0-SNAPSHOT"
|
||||
|
||||
var versions = mapOf(
|
||||
"bwhc-dto-java" to "0.3.0",
|
||||
"hapi-fhir" to "6.10.5",
|
||||
"httpclient5" to "5.2.3",
|
||||
"mockito-kotlin" to "5.3.1",
|
||||
"bwhc-dto-java" to "0.4.0",
|
||||
"mtb-dto" to "0.1.0-SNAPSHOT",
|
||||
"hapi-fhir" to "7.6.0",
|
||||
"mockito-kotlin" to "5.4.0",
|
||||
"archunit" to "1.3.0",
|
||||
// Webjars
|
||||
"webjars-locator" to "0.52",
|
||||
"echarts" to "5.4.3",
|
||||
"htmx.org" to "1.9.12"
|
||||
)
|
||||
|
||||
java {
|
||||
sourceCompatibility = JavaVersion.VERSION_21
|
||||
toolchain {
|
||||
languageVersion = JavaLanguageVersion.of(21)
|
||||
}
|
||||
}
|
||||
|
||||
sourceSets {
|
||||
@ -43,9 +49,18 @@ configurations {
|
||||
compileOnly {
|
||||
extendsFrom(configurations.annotationProcessor.get())
|
||||
}
|
||||
|
||||
all {
|
||||
resolutionStrategy {
|
||||
cacheChangingModulesFor(5, "minutes")
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
repositories {
|
||||
maven {
|
||||
url = uri("https://git.dnpm.dev/api/packages/public-snapshots/maven")
|
||||
}
|
||||
maven {
|
||||
url = uri("https://git.dnpm.dev/api/packages/public/maven")
|
||||
}
|
||||
@ -62,37 +77,46 @@ dependencies {
|
||||
implementation("org.thymeleaf.extras:thymeleaf-extras-springsecurity6")
|
||||
implementation("com.fasterxml.jackson.module:jackson-module-kotlin")
|
||||
implementation("org.springframework.kafka:spring-kafka")
|
||||
implementation("org.flywaydb:flyway-database-postgresql")
|
||||
implementation("org.flywaydb:flyway-mysql")
|
||||
implementation("commons-codec:commons-codec")
|
||||
implementation("io.projectreactor.kotlin:reactor-kotlin-extensions")
|
||||
implementation("de.ukw.ccc:bwhc-dto-java:${versions["bwhc-dto-java"]}")
|
||||
implementation("dev.pcvolkmer.mv64e:mtb-dto:${versions["mtb-dto"]}") { isChanging = true }
|
||||
implementation("ca.uhn.hapi.fhir:hapi-fhir-base:${versions["hapi-fhir"]}")
|
||||
implementation("ca.uhn.hapi.fhir:hapi-fhir-structures-r4:${versions["hapi-fhir"]}")
|
||||
implementation("org.apache.httpcomponents.client5:httpclient5:${versions["httpclient5"]}")
|
||||
implementation("org.apache.httpcomponents.client5:httpclient5")
|
||||
implementation("com.jayway.jsonpath:json-path")
|
||||
implementation("org.webjars:webjars-locator:0.52")
|
||||
implementation("org.webjars:webjars-locator:${versions["webjars-locator"]}")
|
||||
implementation("org.webjars.npm:echarts:${versions["echarts"]}")
|
||||
implementation("org.webjars.npm:htmx.org:${versions["htmx.org"]}")
|
||||
|
||||
runtimeOnly("org.mariadb.jdbc:mariadb-java-client")
|
||||
runtimeOnly("org.postgresql:postgresql")
|
||||
|
||||
developmentOnly("org.springframework.boot:spring-boot-devtools")
|
||||
developmentOnly("org.springframework.boot:spring-boot-docker-compose")
|
||||
|
||||
annotationProcessor("org.springframework.boot:spring-boot-configuration-processor")
|
||||
|
||||
providedRuntime("org.springframework.boot:spring-boot-starter-tomcat")
|
||||
|
||||
testImplementation("org.springframework.boot:spring-boot-starter-test")
|
||||
testImplementation("org.springframework.security:spring-security-test")
|
||||
testImplementation("io.projectreactor:reactor-test")
|
||||
testImplementation("org.mockito.kotlin:mockito-kotlin:${versions["mockito-kotlin"]}")
|
||||
|
||||
integrationTestImplementation("org.testcontainers:junit-jupiter")
|
||||
integrationTestImplementation("org.testcontainers:postgresql")
|
||||
// Override dependency version from org.testcontainers:junit-jupiter - CVE-2024-26308, CVE-2024-25710
|
||||
integrationTestImplementation("org.apache.commons:commons-compress:1.26.1")
|
||||
integrationTestImplementation("com.tngtech.archunit:archunit:${versions["archunit"]}")
|
||||
integrationTestImplementation("org.htmlunit:htmlunit")
|
||||
integrationTestImplementation("org.springframework:spring-webflux")
|
||||
}
|
||||
|
||||
tasks.withType<KotlinCompile> {
|
||||
kotlinOptions {
|
||||
freeCompilerArgs += "-Xjsr305=strict"
|
||||
jvmTarget = "21"
|
||||
compilerOptions {
|
||||
freeCompilerArgs.add("-Xjsr305=strict")
|
||||
jvmTarget.set(JvmTarget.JVM_21)
|
||||
}
|
||||
}
|
||||
|
||||
@ -103,8 +127,9 @@ tasks.withType<Test> {
|
||||
}
|
||||
}
|
||||
|
||||
task<Test>("integrationTest") {
|
||||
tasks.register<Test>("integrationTest") {
|
||||
description = "Runs integration tests"
|
||||
group = "verification"
|
||||
|
||||
testClassesDirs = sourceSets["integrationTest"].output.classesDirs
|
||||
classpath = sourceSets["integrationTest"].runtimeClasspath
|
||||
@ -112,8 +137,24 @@ task<Test>("integrationTest") {
|
||||
shouldRunAfter("test")
|
||||
}
|
||||
|
||||
tasks.register("allTests") {
|
||||
description = "Run all tests"
|
||||
group = JavaBasePlugin.VERIFICATION_GROUP
|
||||
dependsOn(tasks.withType<Test>())
|
||||
}
|
||||
|
||||
tasks.jacocoTestReport {
|
||||
dependsOn("allTests")
|
||||
|
||||
executionData(fileTree(project.rootDir.absolutePath).include("**/build/jacoco/*.exec"))
|
||||
|
||||
reports {
|
||||
xml.required = true
|
||||
}
|
||||
}
|
||||
|
||||
tasks.named<BootBuildImage>("bootBuildImage") {
|
||||
imageName.set("ghcr.io/ccc-mf/etl-processor")
|
||||
imageName.set("ghcr.io/pcvolkmer/etl-processor")
|
||||
|
||||
// Binding for CA Certs
|
||||
bindings.set(listOf(
|
||||
@ -123,7 +164,7 @@ tasks.named<BootBuildImage>("bootBuildImage") {
|
||||
environment.set(environment.get() + mapOf(
|
||||
// Enable this line to embed CA Certs into image on build time
|
||||
//"BP_EMBED_CERTS" to "true",
|
||||
"BP_OCI_SOURCE" to "https://github.com/CCC-MF/etl-processor",
|
||||
"BP_OCI_SOURCE" to "https://github.com/pcvolkmer/etl-processor",
|
||||
"BP_OCI_LICENSES" to "AGPLv3",
|
||||
"BP_OCI_DESCRIPTION" to "ETL Processor for bwHC MTB files"
|
||||
))
|
||||
|
@ -18,6 +18,9 @@ services:
|
||||
APP_KAFKA_GROUP_ID: ${DNPM_KAFKA_GROUP_ID}
|
||||
APP_KAFKA_RESPONSE_TOPIC: ${DNPM_KAFKA_RESPONSE_TOPIC}
|
||||
APP_REST_URI: ${DNPM_BWHC_REST_URI}
|
||||
APP_REST_USERNAME: ${DNPM_BWHC_REST_USERNAME}
|
||||
APP_REST_PASSWORD: ${DNPM_BWHC_REST_PASSWORD}
|
||||
APP_REST_IS_BWHC: ${DNPM_BWHC_REST_IS_BWHC}
|
||||
APP_SECURITY_ADMIN_USER: ${DNPM_ADMIN_USER}
|
||||
APP_SECURITY_ADMIN_PASSWORD: ${DNPM_ADMIN_PASSWORD}
|
||||
SPRING_DATASOURCE_URL: ${DNPM_DATASOURCE_URL}
|
||||
|
@ -28,6 +28,9 @@ DNPM_DATASOURCE_URL=jdbc:mariadb://dnpm-monitor-db:3306/$DNPM_MARIADB_DB
|
||||
## TARGET SYSTEMS CONFIG
|
||||
# in case of direct access to bwhc enter endpoint url here
|
||||
DNPM_BWHC_REST_URI=
|
||||
DNPM_BWHC_REST_USERNAME=
|
||||
DNPM_BWHC_REST_PASSWORD=
|
||||
DNPM_BWHC_REST_IS_BWHC=false
|
||||
|
||||
# produce mtb files to this topic - values 'false' disabling kafka processing
|
||||
DNPM_KAFKA_TOPIC=false
|
||||
|
@ -17,8 +17,9 @@ services:
|
||||
KAFKA_CFG_CONTROLLER_QUORUM_VOTERS: 0@kafka:9093
|
||||
KAFKA_CFG_CONTROLLER_LISTENER_NAMES: CONTROLLER
|
||||
|
||||
## Use AKHQ as Kafka web frontend
|
||||
akhq:
|
||||
image: tchiotludo/akhq:0.21.0
|
||||
image: tchiotludo/akhq:0.25.0
|
||||
environment:
|
||||
AKHQ_CONFIGURATION: |
|
||||
akhq:
|
||||
@ -32,6 +33,8 @@ services:
|
||||
ports:
|
||||
- "8084:8080"
|
||||
|
||||
|
||||
## For use with MariaDB
|
||||
mariadb:
|
||||
image: mariadb:10
|
||||
ports:
|
||||
@ -42,6 +45,7 @@ services:
|
||||
MARIADB_PASSWORD: dev
|
||||
MARIADB_ROOT_PASSWORD: dev
|
||||
|
||||
## For use with Postgres
|
||||
# postgres:
|
||||
# image: postgres:alpine
|
||||
# ports:
|
||||
|
BIN
docs/etl.png
BIN
docs/etl.png
Binary file not shown.
Before Width: | Height: | Size: 120 KiB After Width: | Height: | Size: 115 KiB |
2
gradle/wrapper/gradle-wrapper.properties
vendored
2
gradle/wrapper/gradle-wrapper.properties
vendored
@ -1,6 +1,6 @@
|
||||
distributionBase=GRADLE_USER_HOME
|
||||
distributionPath=wrapper/dists
|
||||
distributionUrl=https\://services.gradle.org/distributions/gradle-8.6-bin.zip
|
||||
distributionUrl=https\://services.gradle.org/distributions/gradle-8.13-bin.zip
|
||||
networkTimeout=10000
|
||||
zipStoreBase=GRADLE_USER_HOME
|
||||
zipStorePath=wrapper/dists
|
||||
|
@ -23,6 +23,7 @@ import com.fasterxml.jackson.databind.ObjectMapper
|
||||
import de.ukw.ccc.bwhc.dto.*
|
||||
import dev.dnpm.etl.processor.monitoring.RequestRepository
|
||||
import dev.dnpm.etl.processor.monitoring.RequestStatus
|
||||
import dev.dnpm.etl.processor.output.BwhcV1MtbFileRequest
|
||||
import dev.dnpm.etl.processor.output.MtbFileSender
|
||||
import org.assertj.core.api.Assertions.assertThat
|
||||
import org.junit.jupiter.api.BeforeEach
|
||||
@ -33,10 +34,10 @@ import org.mockito.kotlin.*
|
||||
import org.springframework.beans.factory.annotation.Autowired
|
||||
import org.springframework.boot.test.autoconfigure.web.servlet.AutoConfigureMockMvc
|
||||
import org.springframework.boot.test.context.SpringBootTest
|
||||
import org.springframework.boot.test.mock.mockito.MockBean
|
||||
import org.springframework.context.ApplicationContext
|
||||
import org.springframework.http.MediaType
|
||||
import org.springframework.test.context.TestPropertySource
|
||||
import org.springframework.test.context.bean.override.mockito.MockitoBean
|
||||
import org.springframework.test.context.junit.jupiter.SpringExtension
|
||||
import org.springframework.test.web.servlet.MockMvc
|
||||
import org.springframework.test.web.servlet.post
|
||||
@ -45,7 +46,7 @@ import org.testcontainers.junit.jupiter.Testcontainers
|
||||
@Testcontainers
|
||||
@ExtendWith(SpringExtension::class)
|
||||
@SpringBootTest
|
||||
@MockBean(MtbFileSender::class)
|
||||
@MockitoBean(types = [MtbFileSender::class])
|
||||
@TestPropertySource(
|
||||
properties = [
|
||||
"app.rest.uri=http://example.com",
|
||||
@ -73,7 +74,7 @@ class EtlProcessorApplicationTests : AbstractTestcontainerTest() {
|
||||
)
|
||||
inner class TransformationTest {
|
||||
|
||||
@MockBean
|
||||
@MockitoBean
|
||||
private lateinit var mtbFileSender: MtbFileSender
|
||||
|
||||
@Autowired
|
||||
@ -91,7 +92,7 @@ class EtlProcessorApplicationTests : AbstractTestcontainerTest() {
|
||||
fun mtbFileIsTransformed() {
|
||||
doAnswer {
|
||||
MtbFileSender.Response(RequestStatus.SUCCESS)
|
||||
}.whenever(mtbFileSender).send(any<MtbFileSender.MtbFileRequest>())
|
||||
}.whenever(mtbFileSender).send(any<BwhcV1MtbFileRequest>())
|
||||
|
||||
val mtbFile = MtbFile.builder()
|
||||
.withPatient(
|
||||
@ -134,9 +135,9 @@ class EtlProcessorApplicationTests : AbstractTestcontainerTest() {
|
||||
}
|
||||
}
|
||||
|
||||
val captor = argumentCaptor<MtbFileSender.MtbFileRequest>()
|
||||
val captor = argumentCaptor<BwhcV1MtbFileRequest>()
|
||||
verify(mtbFileSender).send(captor.capture())
|
||||
assertThat(captor.firstValue.mtbFile.diagnoses).hasSize(1).allMatch { diagnosis ->
|
||||
assertThat(captor.firstValue.content.diagnoses).hasSize(1).allMatch { diagnosis ->
|
||||
diagnosis.icd10.version == "2014"
|
||||
}
|
||||
}
|
||||
|
@ -0,0 +1,73 @@
|
||||
package dev.dnpm.etl.processor
|
||||
|
||||
import com.tngtech.archunit.core.domain.JavaClasses
|
||||
import com.tngtech.archunit.core.importer.ClassFileImporter
|
||||
import com.tngtech.archunit.lang.syntax.ArchRuleDefinition.classes
|
||||
import com.tngtech.archunit.lang.syntax.ArchRuleDefinition.noClasses
|
||||
import org.junit.jupiter.api.BeforeEach
|
||||
import org.junit.jupiter.api.Test
|
||||
import org.springframework.data.repository.Repository
|
||||
|
||||
class EtlProcessorArchTest {
|
||||
|
||||
private lateinit var noTestClasses: JavaClasses
|
||||
|
||||
@BeforeEach
|
||||
fun setUp() {
|
||||
this.noTestClasses = ClassFileImporter()
|
||||
.withImportOption { !(it.contains("/test/") || it.contains("/integrationTest/")) }
|
||||
.importPackages("dev.dnpm.etl.processor")
|
||||
}
|
||||
|
||||
@Test
|
||||
fun noClassesInInputPackageShouldDependOnMonitoringPackage() {
|
||||
val rule = noClasses()
|
||||
.that()
|
||||
.resideInAPackage("..input")
|
||||
.should().dependOnClassesThat()
|
||||
.resideInAnyPackage("..monitoring")
|
||||
|
||||
rule.check(noTestClasses)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun noClassesInInputPackageShouldDependOnRepositories() {
|
||||
val rule = noClasses()
|
||||
.that()
|
||||
.resideInAPackage("..input")
|
||||
.should().dependOnClassesThat().haveSimpleNameEndingWith("Repository")
|
||||
|
||||
rule.check(noTestClasses)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun noClassesInOutputPackageShouldDependOnRepositories() {
|
||||
val rule = noClasses()
|
||||
.that()
|
||||
.resideInAPackage("..output")
|
||||
.should().dependOnClassesThat().haveSimpleNameEndingWith("Repository")
|
||||
|
||||
rule.check(noTestClasses)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun noClassesInWebPackageShouldDependOnRepositories() {
|
||||
val rule = noClasses()
|
||||
.that()
|
||||
.resideInAPackage("..web")
|
||||
.should().dependOnClassesThat().haveSimpleNameEndingWith("Repository")
|
||||
|
||||
rule.check(noTestClasses)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun repositoryClassNamesShouldEndWithRepository() {
|
||||
val rule = classes()
|
||||
.that()
|
||||
.areInterfaces().and().areAssignableTo(Repository::class.java)
|
||||
.should().haveSimpleNameEndingWith("Repository")
|
||||
|
||||
rule.check(noTestClasses)
|
||||
}
|
||||
|
||||
}
|
@ -26,9 +26,9 @@ import dev.dnpm.etl.processor.output.KafkaMtbFileSender
|
||||
import dev.dnpm.etl.processor.output.RestMtbFileSender
|
||||
import dev.dnpm.etl.processor.pseudonym.AnonymizingGenerator
|
||||
import dev.dnpm.etl.processor.pseudonym.GpasPseudonymGenerator
|
||||
import dev.dnpm.etl.processor.security.TokenRepository
|
||||
import dev.dnpm.etl.processor.security.TokenService
|
||||
import dev.dnpm.etl.processor.services.RequestProcessor
|
||||
import dev.dnpm.etl.processor.services.TokenRepository
|
||||
import dev.dnpm.etl.processor.services.TokenService
|
||||
import org.assertj.core.api.Assertions.assertThat
|
||||
import org.junit.jupiter.api.Nested
|
||||
import org.junit.jupiter.api.Test
|
||||
@ -36,24 +36,25 @@ import org.junit.jupiter.api.assertThrows
|
||||
import org.springframework.beans.factory.NoSuchBeanDefinitionException
|
||||
import org.springframework.boot.autoconfigure.kafka.KafkaAutoConfiguration
|
||||
import org.springframework.boot.test.context.SpringBootTest
|
||||
import org.springframework.boot.test.mock.mockito.MockBean
|
||||
import org.springframework.boot.test.mock.mockito.MockBeans
|
||||
import org.springframework.context.ApplicationContext
|
||||
import org.springframework.retry.support.RetryTemplate
|
||||
import org.springframework.security.crypto.password.PasswordEncoder
|
||||
import org.springframework.security.provisioning.InMemoryUserDetailsManager
|
||||
import org.springframework.test.context.ContextConfiguration
|
||||
import org.springframework.test.context.TestPropertySource
|
||||
import org.springframework.test.context.bean.override.mockito.MockitoBean
|
||||
|
||||
@SpringBootTest
|
||||
@ContextConfiguration(classes = [
|
||||
@ContextConfiguration(
|
||||
classes = [
|
||||
AppConfiguration::class,
|
||||
AppSecurityConfiguration::class,
|
||||
KafkaAutoConfiguration::class,
|
||||
AppKafkaConfiguration::class,
|
||||
AppRestConfiguration::class
|
||||
])
|
||||
@MockBean(ObjectMapper::class)
|
||||
]
|
||||
)
|
||||
@MockitoBean(types = [ObjectMapper::class])
|
||||
@TestPropertySource(
|
||||
properties = [
|
||||
"app.pseudonymize.generator=BUILDIN",
|
||||
@ -86,7 +87,7 @@ class AppConfigurationTest {
|
||||
"app.kafka.group-id=test"
|
||||
]
|
||||
)
|
||||
@MockBean(RequestRepository::class)
|
||||
@MockitoBean(types = [RequestRepository::class])
|
||||
inner class AppConfigurationKafkaTest(private val context: ApplicationContext) {
|
||||
|
||||
@Test
|
||||
@ -145,7 +146,7 @@ class AppConfigurationTest {
|
||||
"app.kafka.group-id=test"
|
||||
]
|
||||
)
|
||||
@MockBean(RequestProcessor::class)
|
||||
@MockitoBean(types = [RequestProcessor::class])
|
||||
inner class AppConfigurationUsingKafkaInputTest(private val context: ApplicationContext) {
|
||||
|
||||
@Test
|
||||
@ -181,40 +182,7 @@ class AppConfigurationTest {
|
||||
@Nested
|
||||
@TestPropertySource(
|
||||
properties = [
|
||||
"app.pseudonymize.generator=",
|
||||
"app.pseudonymizer=buildin",
|
||||
]
|
||||
)
|
||||
inner class AppConfigurationPseudonymizerBuildinTest(private val context: ApplicationContext) {
|
||||
|
||||
@Test
|
||||
fun shouldUseConfiguredGenerator() {
|
||||
assertThat(context.getBean(AnonymizingGenerator::class.java)).isNotNull
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
@Nested
|
||||
@TestPropertySource(
|
||||
properties = [
|
||||
"app.pseudonymize.generator=",
|
||||
"app.pseudonymizer=gpas",
|
||||
]
|
||||
)
|
||||
inner class AppConfigurationPseudonymizerGpasTest(private val context: ApplicationContext) {
|
||||
|
||||
@Test
|
||||
fun shouldUseConfiguredGenerator() {
|
||||
assertThat(context.getBean(GpasPseudonymGenerator::class.java)).isNotNull
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
@Nested
|
||||
@TestPropertySource(
|
||||
properties = [
|
||||
"app.pseudonymize.generator=buildin",
|
||||
"app.pseudonymizer=",
|
||||
"app.pseudonymize.generator=buildin"
|
||||
]
|
||||
)
|
||||
inner class AppConfigurationPseudonymizeGeneratorBuildinTest(private val context: ApplicationContext) {
|
||||
@ -229,8 +197,7 @@ class AppConfigurationTest {
|
||||
@Nested
|
||||
@TestPropertySource(
|
||||
properties = [
|
||||
"app.pseudonymize.generator=gpas",
|
||||
"app.pseudonymizer=",
|
||||
"app.pseudonymize.generator=gpas"
|
||||
]
|
||||
)
|
||||
inner class AppConfigurationPseudonymizeGeneratorGpasTest(private val context: ApplicationContext) {
|
||||
@ -248,11 +215,13 @@ class AppConfigurationTest {
|
||||
"app.security.enable-tokens=true"
|
||||
]
|
||||
)
|
||||
@MockBeans(value = [
|
||||
MockBean(InMemoryUserDetailsManager::class),
|
||||
MockBean(PasswordEncoder::class),
|
||||
MockBean(TokenRepository::class)
|
||||
])
|
||||
@MockitoBean(
|
||||
types = [
|
||||
InMemoryUserDetailsManager::class,
|
||||
PasswordEncoder::class,
|
||||
TokenRepository::class
|
||||
]
|
||||
)
|
||||
inner class AppConfigurationTokenEnabledTest(private val context: ApplicationContext) {
|
||||
|
||||
@Test
|
||||
@ -263,11 +232,13 @@ class AppConfigurationTest {
|
||||
}
|
||||
|
||||
@Nested
|
||||
@MockBeans(value = [
|
||||
MockBean(InMemoryUserDetailsManager::class),
|
||||
MockBean(PasswordEncoder::class),
|
||||
MockBean(TokenRepository::class)
|
||||
])
|
||||
@MockitoBean(
|
||||
types = [
|
||||
InMemoryUserDetailsManager::class,
|
||||
PasswordEncoder::class,
|
||||
TokenRepository::class
|
||||
]
|
||||
)
|
||||
inner class AppConfigurationTokenDisabledTest(private val context: ApplicationContext) {
|
||||
|
||||
@Test
|
||||
|
30
src/integrationTest/kotlin/dev/dnpm/etl/processor/helpers.kt
Normal file
30
src/integrationTest/kotlin/dev/dnpm/etl/processor/helpers.kt
Normal file
@ -0,0 +1,30 @@
|
||||
/*
|
||||
* This file is part of ETL-Processor
|
||||
*
|
||||
* Copyright (c) 2024 Comprehensive Cancer Center Mainfranken, Datenintegrationszentrum Philipps-Universität Marburg and Contributors
|
||||
*
|
||||
* This program is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
* by the Free Software Foundation, either version 3 of the License, or
|
||||
* (at your option) any later version.
|
||||
*
|
||||
* This program is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
* GNU Affero General Public License for more details.
|
||||
*
|
||||
* You should have received a copy of the GNU Affero General Public License
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
*/
|
||||
|
||||
package dev.dnpm.etl.processor
|
||||
|
||||
import org.mockito.ArgumentMatchers
|
||||
|
||||
@Suppress("UNCHECKED_CAST")
|
||||
inline fun <reified T> anyValueClass(): T {
|
||||
val unboxedClass = T::class.java.declaredFields.first().type
|
||||
return ArgumentMatchers.any(unboxedClass as Class<T>)
|
||||
?: T::class.java.getDeclaredMethod("box-impl", unboxedClass)
|
||||
.invoke(null, null) as T
|
||||
}
|
@ -21,13 +21,15 @@ package dev.dnpm.etl.processor.input
|
||||
|
||||
import com.fasterxml.jackson.databind.ObjectMapper
|
||||
import de.ukw.ccc.bwhc.dto.*
|
||||
import dev.dnpm.etl.processor.anyValueClass
|
||||
import dev.dnpm.etl.processor.config.AppSecurityConfiguration
|
||||
import dev.dnpm.etl.processor.security.TokenRepository
|
||||
import dev.dnpm.etl.processor.security.UserRoleRepository
|
||||
import dev.dnpm.etl.processor.services.RequestProcessor
|
||||
import dev.dnpm.etl.processor.services.TokenRepository
|
||||
import org.junit.jupiter.api.BeforeEach
|
||||
import org.junit.jupiter.api.Nested
|
||||
import org.junit.jupiter.api.Test
|
||||
import org.junit.jupiter.api.extension.ExtendWith
|
||||
import org.mockito.ArgumentMatchers.anyString
|
||||
import org.mockito.junit.jupiter.MockitoExtension
|
||||
import org.mockito.kotlin.any
|
||||
import org.mockito.kotlin.never
|
||||
@ -35,12 +37,13 @@ import org.mockito.kotlin.times
|
||||
import org.mockito.kotlin.verify
|
||||
import org.springframework.beans.factory.annotation.Autowired
|
||||
import org.springframework.boot.test.autoconfigure.web.servlet.WebMvcTest
|
||||
import org.springframework.boot.test.mock.mockito.MockBean
|
||||
import org.springframework.http.MediaType
|
||||
import org.springframework.security.oauth2.client.registration.ClientRegistrationRepository
|
||||
import org.springframework.security.test.web.servlet.request.SecurityMockMvcRequestPostProcessors.anonymous
|
||||
import org.springframework.security.test.web.servlet.request.SecurityMockMvcRequestPostProcessors.user
|
||||
import org.springframework.test.context.ContextConfiguration
|
||||
import org.springframework.test.context.TestPropertySource
|
||||
import org.springframework.test.context.bean.override.mockito.MockitoBean
|
||||
import org.springframework.test.context.junit.jupiter.SpringExtension
|
||||
import org.springframework.test.web.servlet.MockMvc
|
||||
import org.springframework.test.web.servlet.delete
|
||||
@ -54,7 +57,7 @@ import org.springframework.test.web.servlet.post
|
||||
AppSecurityConfiguration::class
|
||||
]
|
||||
)
|
||||
@MockBean(TokenRepository::class, RequestProcessor::class)
|
||||
@MockitoBean(types = [TokenRepository::class, RequestProcessor::class])
|
||||
@TestPropertySource(
|
||||
properties = [
|
||||
"app.pseudonymize.generator=BUILDIN",
|
||||
@ -88,7 +91,20 @@ class MtbFileRestControllerTest {
|
||||
status { isAccepted() }
|
||||
}
|
||||
|
||||
verify(requestProcessor, times(1)).processMtbFile(any())
|
||||
verify(requestProcessor, times(1)).processMtbFile(any<MtbFile>())
|
||||
}
|
||||
|
||||
@Test
|
||||
fun testShouldGrantPermissionToSendMtbFileToAdminUser() {
|
||||
mockMvc.post("/mtbfile") {
|
||||
with(user("onkostarserver").roles("ADMIN"))
|
||||
contentType = MediaType.APPLICATION_JSON
|
||||
content = ObjectMapper().writeValueAsString(mtbFile)
|
||||
}.andExpect {
|
||||
status { isAccepted() }
|
||||
}
|
||||
|
||||
verify(requestProcessor, times(1)).processMtbFile(any<MtbFile>())
|
||||
}
|
||||
|
||||
@Test
|
||||
@ -101,7 +117,20 @@ class MtbFileRestControllerTest {
|
||||
status { isUnauthorized() }
|
||||
}
|
||||
|
||||
verify(requestProcessor, never()).processMtbFile(any())
|
||||
verify(requestProcessor, never()).processMtbFile(any<MtbFile>())
|
||||
}
|
||||
|
||||
@Test
|
||||
fun testShouldDenyPermissionToSendMtbFileForUser() {
|
||||
mockMvc.post("/mtbfile") {
|
||||
with(user("fakeuser").roles("USER"))
|
||||
contentType = MediaType.APPLICATION_JSON
|
||||
content = ObjectMapper().writeValueAsString(mtbFile)
|
||||
}.andExpect {
|
||||
status { isForbidden() }
|
||||
}
|
||||
|
||||
verify(requestProcessor, never()).processMtbFile(any<MtbFile>())
|
||||
}
|
||||
|
||||
@Test
|
||||
@ -112,7 +141,7 @@ class MtbFileRestControllerTest {
|
||||
status { isAccepted() }
|
||||
}
|
||||
|
||||
verify(requestProcessor, times(1)).processDeletion(anyString())
|
||||
verify(requestProcessor, times(1)).processDeletion(anyValueClass())
|
||||
}
|
||||
|
||||
@Test
|
||||
@ -123,7 +152,46 @@ class MtbFileRestControllerTest {
|
||||
status { isUnauthorized() }
|
||||
}
|
||||
|
||||
verify(requestProcessor, never()).processDeletion(anyString())
|
||||
verify(requestProcessor, never()).processDeletion(anyValueClass())
|
||||
}
|
||||
|
||||
@Nested
|
||||
@MockitoBean(types = [UserRoleRepository::class, ClientRegistrationRepository::class])
|
||||
@TestPropertySource(
|
||||
properties = [
|
||||
"app.pseudonymize.generator=BUILDIN",
|
||||
"app.security.admin-user=admin",
|
||||
"app.security.admin-password={noop}very-secret",
|
||||
"app.security.enable-tokens=true",
|
||||
"app.security.enable-oidc=true"
|
||||
]
|
||||
)
|
||||
inner class WithOidcEnabled {
|
||||
@Test
|
||||
fun testShouldGrantPermissionToSendMtbFileToAdminUser() {
|
||||
mockMvc.post("/mtbfile") {
|
||||
with(user("onkostarserver").roles("ADMIN"))
|
||||
contentType = MediaType.APPLICATION_JSON
|
||||
content = ObjectMapper().writeValueAsString(mtbFile)
|
||||
}.andExpect {
|
||||
status { isAccepted() }
|
||||
}
|
||||
|
||||
verify(requestProcessor, times(1)).processMtbFile(any<MtbFile>())
|
||||
}
|
||||
|
||||
@Test
|
||||
fun testShouldGrantPermissionToSendMtbFileToUser() {
|
||||
mockMvc.post("/mtbfile") {
|
||||
with(user("onkostarserver").roles("USER"))
|
||||
contentType = MediaType.APPLICATION_JSON
|
||||
content = ObjectMapper().writeValueAsString(mtbFile)
|
||||
}.andExpect {
|
||||
status { isAccepted() }
|
||||
}
|
||||
|
||||
verify(requestProcessor, times(1)).processMtbFile(any<MtbFile>())
|
||||
}
|
||||
}
|
||||
|
||||
companion object {
|
||||
|
@ -0,0 +1,75 @@
|
||||
/*
|
||||
* This file is part of ETL-Processor
|
||||
*
|
||||
* Copyright (c) 2024 Comprehensive Cancer Center Mainfranken, Datenintegrationszentrum Philipps-Universität Marburg and Contributors
|
||||
*
|
||||
* This program is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
* by the Free Software Foundation, either version 3 of the License, or
|
||||
* (at your option) any later version.
|
||||
*
|
||||
* This program is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
* GNU Affero General Public License for more details.
|
||||
*
|
||||
* You should have received a copy of the GNU Affero General Public License
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
*/
|
||||
|
||||
package dev.dnpm.etl.processor.monitoring
|
||||
|
||||
import dev.dnpm.etl.processor.*
|
||||
import dev.dnpm.etl.processor.output.MtbFileSender
|
||||
import org.junit.jupiter.api.BeforeEach
|
||||
import org.junit.jupiter.api.Test
|
||||
import org.junit.jupiter.api.extension.ExtendWith
|
||||
import org.springframework.beans.factory.annotation.Autowired
|
||||
import org.springframework.boot.test.autoconfigure.data.jdbc.DataJdbcTest
|
||||
import org.springframework.boot.test.autoconfigure.jdbc.AutoConfigureTestDatabase
|
||||
import org.springframework.test.context.TestPropertySource
|
||||
import org.springframework.test.context.bean.override.mockito.MockitoBean
|
||||
import org.springframework.test.context.junit.jupiter.SpringExtension
|
||||
import org.springframework.transaction.annotation.Transactional
|
||||
import org.testcontainers.junit.jupiter.Testcontainers
|
||||
import java.time.Instant
|
||||
|
||||
@Testcontainers
|
||||
@ExtendWith(SpringExtension::class)
|
||||
@DataJdbcTest
|
||||
@AutoConfigureTestDatabase(replace = AutoConfigureTestDatabase.Replace.NONE)
|
||||
@Transactional
|
||||
@MockitoBean(types = [MtbFileSender::class])
|
||||
@TestPropertySource(
|
||||
properties = [
|
||||
"app.pseudonymize.generator=buildin",
|
||||
"app.rest.uri=http://example.com"
|
||||
]
|
||||
)
|
||||
class RequestRepositoryTest : AbstractTestcontainerTest() {
|
||||
|
||||
private lateinit var requestRepository: RequestRepository
|
||||
|
||||
@BeforeEach
|
||||
fun setUp(
|
||||
@Autowired requestRepository: RequestRepository
|
||||
) {
|
||||
this.requestRepository = requestRepository
|
||||
}
|
||||
|
||||
@Test
|
||||
fun shouldSaveRequest() {
|
||||
val request = Request(
|
||||
randomRequestId(),
|
||||
PatientPseudonym("TEST_12345678901"),
|
||||
PatientId("P1"),
|
||||
Fingerprint("0123456789abcdef1"),
|
||||
RequestType.MTB_FILE,
|
||||
RequestStatus.WARNING,
|
||||
Instant.parse("2023-07-07T00:00:00Z")
|
||||
)
|
||||
|
||||
requestRepository.save(request)
|
||||
}
|
||||
|
||||
}
|
@ -0,0 +1,135 @@
|
||||
/*
|
||||
* This file is part of ETL-Processor
|
||||
*
|
||||
* Copyright (c) 2025 Comprehensive Cancer Center Mainfranken, Datenintegrationszentrum Philipps-Universität Marburg and Contributors
|
||||
*
|
||||
* This program is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
* by the Free Software Foundation, either version 3 of the License, or
|
||||
* (at your option) any later version.
|
||||
*
|
||||
* This program is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
* GNU Affero General Public License for more details.
|
||||
*
|
||||
* You should have received a copy of the GNU Affero General Public License
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
*/
|
||||
|
||||
package dev.dnpm.etl.processor.pseudonym
|
||||
|
||||
import dev.dnpm.etl.processor.config.GPasConfigProperties
|
||||
import org.assertj.core.api.Assertions.assertThat
|
||||
import org.junit.jupiter.api.BeforeEach
|
||||
import org.junit.jupiter.api.Test
|
||||
import org.junit.jupiter.api.assertThrows
|
||||
import org.springframework.http.HttpHeaders
|
||||
import org.springframework.http.HttpMethod
|
||||
import org.springframework.http.HttpStatus
|
||||
import org.springframework.retry.policy.SimpleRetryPolicy
|
||||
import org.springframework.retry.support.RetryTemplateBuilder
|
||||
import org.springframework.test.web.client.MockRestServiceServer
|
||||
import org.springframework.test.web.client.match.MockRestRequestMatchers.method
|
||||
import org.springframework.test.web.client.match.MockRestRequestMatchers.requestTo
|
||||
import org.springframework.test.web.client.response.MockRestResponseCreators.withException
|
||||
import org.springframework.test.web.client.response.MockRestResponseCreators.withStatus
|
||||
import org.springframework.web.client.RestTemplate
|
||||
import java.io.IOException
|
||||
|
||||
class GpasPseudonymGeneratorTest {
|
||||
|
||||
private lateinit var mockRestServiceServer: MockRestServiceServer
|
||||
private lateinit var generator: GpasPseudonymGenerator
|
||||
private lateinit var restTemplate: RestTemplate
|
||||
|
||||
@BeforeEach
|
||||
fun setup() {
|
||||
val retryTemplate = RetryTemplateBuilder().customPolicy(SimpleRetryPolicy(1)).build()
|
||||
val gPasConfigProperties = GPasConfigProperties(
|
||||
"https://localhost/ttp-fhir/fhir/gpas/\$pseudonymizeAllowCreate",
|
||||
"test",
|
||||
null,
|
||||
null
|
||||
)
|
||||
|
||||
this.restTemplate = RestTemplate()
|
||||
this.mockRestServiceServer = MockRestServiceServer.createServer(restTemplate)
|
||||
this.generator = GpasPseudonymGenerator(gPasConfigProperties, retryTemplate, restTemplate)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun shouldReturnExpectedPseudonym() {
|
||||
this.mockRestServiceServer.expect {
|
||||
method(HttpMethod.POST)
|
||||
requestTo("https://localhost/ttp-fhir/fhir/gpas/\$pseudonymizeAllowCreate")
|
||||
}.andRespond {
|
||||
withStatus(HttpStatus.OK).body(getDummyResponseBody("1234", "test", "test1234ABCDEF567890"))
|
||||
.createResponse(it)
|
||||
}
|
||||
|
||||
assertThat(this.generator.generate("ID1234")).isEqualTo("test1234ABCDEF567890")
|
||||
}
|
||||
|
||||
@Test
|
||||
fun shouldThrowExceptionIfGpasNotAvailable() {
|
||||
this.mockRestServiceServer.expect {
|
||||
method(HttpMethod.POST)
|
||||
requestTo("https://localhost/ttp-fhir/fhir/gpas/\$pseudonymizeAllowCreate")
|
||||
}.andRespond {
|
||||
withException(IOException("Simulated IO error")).createResponse(it)
|
||||
}
|
||||
|
||||
assertThrows<PseudonymRequestFailed> { this.generator.generate("ID1234") }
|
||||
}
|
||||
|
||||
@Test
|
||||
fun shouldThrowExceptionIfGpasDoesNotReturn2xxResponse() {
|
||||
this.mockRestServiceServer.expect {
|
||||
method(HttpMethod.POST)
|
||||
requestTo("https://localhost/ttp-fhir/fhir/gpas/\$pseudonymizeAllowCreate")
|
||||
}.andRespond {
|
||||
withStatus(HttpStatus.FOUND)
|
||||
.header(HttpHeaders.LOCATION, "https://localhost/ttp-fhir/fhir/gpas/\$pseudonymizeAllowCreate")
|
||||
.createResponse(it)
|
||||
}
|
||||
|
||||
assertThrows<PseudonymRequestFailed> { this.generator.generate("ID1234") }
|
||||
}
|
||||
|
||||
companion object {
|
||||
|
||||
fun getDummyResponseBody(original: String, target: String, pseudonym: String) = """{
|
||||
"resourceType": "Parameters",
|
||||
"parameter": [
|
||||
{
|
||||
"name": "pseudonym",
|
||||
"part": [
|
||||
{
|
||||
"name": "original",
|
||||
"valueIdentifier": {
|
||||
"system": "https://ths-greifswald.de/gpas",
|
||||
"value": "$original"
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "target",
|
||||
"valueIdentifier": {
|
||||
"system": "https://ths-greifswald.de/gpas",
|
||||
"value": "$target"
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "pseudonym",
|
||||
"valueIdentifier": {
|
||||
"system": "https://ths-greifswald.de/gpas",
|
||||
"value": "$pseudonym"
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
}""".trimIndent()
|
||||
|
||||
}
|
||||
}
|
@ -19,7 +19,7 @@
|
||||
|
||||
package dev.dnpm.etl.processor.services
|
||||
|
||||
import dev.dnpm.etl.processor.AbstractTestcontainerTest
|
||||
import dev.dnpm.etl.processor.*
|
||||
import dev.dnpm.etl.processor.monitoring.Request
|
||||
import dev.dnpm.etl.processor.monitoring.RequestRepository
|
||||
import dev.dnpm.etl.processor.monitoring.RequestStatus
|
||||
@ -31,19 +31,18 @@ import org.junit.jupiter.api.Test
|
||||
import org.junit.jupiter.api.extension.ExtendWith
|
||||
import org.springframework.beans.factory.annotation.Autowired
|
||||
import org.springframework.boot.test.context.SpringBootTest
|
||||
import org.springframework.boot.test.mock.mockito.MockBean
|
||||
import org.springframework.test.context.TestPropertySource
|
||||
import org.springframework.test.context.bean.override.mockito.MockitoBean
|
||||
import org.springframework.test.context.junit.jupiter.SpringExtension
|
||||
import org.springframework.transaction.annotation.Transactional
|
||||
import org.testcontainers.junit.jupiter.Testcontainers
|
||||
import java.time.Instant
|
||||
import java.util.*
|
||||
|
||||
@Testcontainers
|
||||
@ExtendWith(SpringExtension::class)
|
||||
@SpringBootTest
|
||||
@Transactional
|
||||
@MockBean(MtbFileSender::class)
|
||||
@MockitoBean(types = [MtbFileSender::class])
|
||||
@TestPropertySource(
|
||||
properties = [
|
||||
"app.pseudonymize.generator=buildin",
|
||||
@ -66,7 +65,7 @@ class RequestServiceIntegrationTest : AbstractTestcontainerTest() {
|
||||
|
||||
@Test
|
||||
fun shouldResultInEmptyRequestList() {
|
||||
val actual = requestService.allRequestsByPatientPseudonym("TEST_12345678901")
|
||||
val actual = requestService.allRequestsByPatientPseudonym(TEST_PATIENT_PSEUDONYM)
|
||||
|
||||
assertThat(actual).isEmpty()
|
||||
}
|
||||
@ -76,33 +75,33 @@ class RequestServiceIntegrationTest : AbstractTestcontainerTest() {
|
||||
this.requestRepository.saveAll(
|
||||
listOf(
|
||||
Request(
|
||||
uuid = UUID.randomUUID().toString(),
|
||||
patientId = "TEST_12345678901",
|
||||
pid = "P1",
|
||||
fingerprint = "0123456789abcdef1",
|
||||
type = RequestType.MTB_FILE,
|
||||
status = RequestStatus.SUCCESS,
|
||||
processedAt = Instant.parse("2023-07-07T02:00:00Z")
|
||||
randomRequestId(),
|
||||
PatientPseudonym("TEST_12345678901"),
|
||||
PatientId("P1"),
|
||||
Fingerprint("0123456789abcdef1"),
|
||||
RequestType.MTB_FILE,
|
||||
RequestStatus.SUCCESS,
|
||||
Instant.parse("2023-07-07T02:00:00Z")
|
||||
),
|
||||
// Should be ignored - wrong patient ID -->
|
||||
Request(
|
||||
uuid = UUID.randomUUID().toString(),
|
||||
patientId = "TEST_12345678902",
|
||||
pid = "P2",
|
||||
fingerprint = "0123456789abcdef2",
|
||||
type = RequestType.MTB_FILE,
|
||||
status = RequestStatus.WARNING,
|
||||
processedAt = Instant.parse("2023-08-08T00:00:00Z")
|
||||
randomRequestId(),
|
||||
PatientPseudonym("TEST_12345678902"),
|
||||
PatientId("P2"),
|
||||
Fingerprint("0123456789abcdef2"),
|
||||
RequestType.MTB_FILE,
|
||||
RequestStatus.WARNING,
|
||||
Instant.parse("2023-08-08T00:00:00Z")
|
||||
),
|
||||
// <--
|
||||
Request(
|
||||
uuid = UUID.randomUUID().toString(),
|
||||
patientId = "TEST_12345678901",
|
||||
pid = "P2",
|
||||
fingerprint = "0123456789abcdee1",
|
||||
type = RequestType.DELETE,
|
||||
status = RequestStatus.SUCCESS,
|
||||
processedAt = Instant.parse("2023-08-08T02:00:00Z")
|
||||
randomRequestId(),
|
||||
PatientPseudonym("TEST_12345678901"),
|
||||
PatientId("P2"),
|
||||
Fingerprint("0123456789abcdee1"),
|
||||
RequestType.DELETE,
|
||||
RequestStatus.SUCCESS,
|
||||
Instant.parse("2023-08-08T02:00:00Z")
|
||||
)
|
||||
)
|
||||
)
|
||||
@ -112,18 +111,18 @@ class RequestServiceIntegrationTest : AbstractTestcontainerTest() {
|
||||
fun shouldResultInSortedRequestList() {
|
||||
setupTestData()
|
||||
|
||||
val actual = requestService.allRequestsByPatientPseudonym("TEST_12345678901")
|
||||
val actual = requestService.allRequestsByPatientPseudonym(TEST_PATIENT_PSEUDONYM)
|
||||
|
||||
assertThat(actual).hasSize(2)
|
||||
assertThat(actual[0].fingerprint).isEqualTo("0123456789abcdee1")
|
||||
assertThat(actual[1].fingerprint).isEqualTo("0123456789abcdef1")
|
||||
assertThat(actual[0].fingerprint).isEqualTo(Fingerprint("0123456789abcdee1"))
|
||||
assertThat(actual[1].fingerprint).isEqualTo(Fingerprint("0123456789abcdef1"))
|
||||
}
|
||||
|
||||
@Test
|
||||
fun shouldReturnDeleteRequestAsLastRequest() {
|
||||
setupTestData()
|
||||
|
||||
val actual = requestService.isLastRequestWithKnownStatusDeletion("TEST_12345678901")
|
||||
val actual = requestService.isLastRequestWithKnownStatusDeletion(TEST_PATIENT_PSEUDONYM)
|
||||
|
||||
assertThat(actual).isTrue()
|
||||
}
|
||||
@ -132,10 +131,14 @@ class RequestServiceIntegrationTest : AbstractTestcontainerTest() {
|
||||
fun shouldReturnLastMtbFileRequest() {
|
||||
setupTestData()
|
||||
|
||||
val actual = requestService.lastMtbFileRequestForPatientPseudonym("TEST_12345678901")
|
||||
val actual = requestService.lastMtbFileRequestForPatientPseudonym(TEST_PATIENT_PSEUDONYM)
|
||||
|
||||
assertThat(actual).isNotNull
|
||||
assertThat(actual?.fingerprint).isEqualTo("0123456789abcdef1")
|
||||
assertThat(actual?.fingerprint).isEqualTo(Fingerprint("0123456789abcdef1"))
|
||||
}
|
||||
|
||||
companion object {
|
||||
val TEST_PATIENT_PSEUDONYM = PatientPseudonym("TEST_12345678901")
|
||||
}
|
||||
|
||||
}
|
@ -21,30 +21,49 @@ package dev.dnpm.etl.processor.web
|
||||
|
||||
import dev.dnpm.etl.processor.config.AppConfiguration
|
||||
import dev.dnpm.etl.processor.config.AppSecurityConfiguration
|
||||
import dev.dnpm.etl.processor.monitoring.ConnectionCheckService
|
||||
import dev.dnpm.etl.processor.monitoring.ConnectionCheckResult
|
||||
import dev.dnpm.etl.processor.monitoring.GPasConnectionCheckService
|
||||
import dev.dnpm.etl.processor.monitoring.RestConnectionCheckService
|
||||
import dev.dnpm.etl.processor.output.MtbFileSender
|
||||
import dev.dnpm.etl.processor.pseudonym.Generator
|
||||
import dev.dnpm.etl.processor.security.Role
|
||||
import dev.dnpm.etl.processor.security.TokenService
|
||||
import dev.dnpm.etl.processor.security.UserRoleService
|
||||
import dev.dnpm.etl.processor.services.RequestProcessor
|
||||
import dev.dnpm.etl.processor.services.TokenRepository
|
||||
import dev.dnpm.etl.processor.services.TransformationService
|
||||
import org.assertj.core.api.Assertions.assertThat
|
||||
import org.htmlunit.WebClient
|
||||
import org.htmlunit.html.HtmlPage
|
||||
import org.junit.jupiter.api.BeforeEach
|
||||
import org.junit.jupiter.api.Nested
|
||||
import org.junit.jupiter.api.Test
|
||||
import org.junit.jupiter.api.extension.ExtendWith
|
||||
import org.mockito.ArgumentMatchers.anyString
|
||||
import org.mockito.junit.jupiter.MockitoExtension
|
||||
import org.mockito.kotlin.argumentCaptor
|
||||
import org.mockito.kotlin.times
|
||||
import org.mockito.kotlin.verify
|
||||
import org.mockito.kotlin.whenever
|
||||
import org.springframework.beans.factory.annotation.Autowired
|
||||
import org.springframework.boot.test.autoconfigure.web.servlet.WebMvcTest
|
||||
import org.springframework.boot.test.mock.mockito.MockBean
|
||||
import org.springframework.http.HttpHeaders
|
||||
import org.springframework.http.MediaType
|
||||
import org.springframework.http.MediaType.TEXT_EVENT_STREAM
|
||||
import org.springframework.security.test.context.support.WithMockUser
|
||||
import org.springframework.security.test.web.servlet.request.SecurityMockMvcRequestPostProcessors.anonymous
|
||||
import org.springframework.security.test.web.servlet.request.SecurityMockMvcRequestPostProcessors.user
|
||||
import org.springframework.test.context.ContextConfiguration
|
||||
import org.springframework.test.context.TestPropertySource
|
||||
import org.springframework.test.context.bean.override.mockito.MockitoBean
|
||||
import org.springframework.test.context.junit.jupiter.SpringExtension
|
||||
import org.springframework.test.web.servlet.MockMvc
|
||||
import org.springframework.test.web.servlet.get
|
||||
import org.springframework.test.web.reactive.server.WebTestClient
|
||||
import org.springframework.test.web.servlet.*
|
||||
import org.springframework.test.web.servlet.client.MockMvcWebTestClient
|
||||
import org.springframework.test.web.servlet.htmlunit.MockMvcWebClientBuilder
|
||||
import org.springframework.web.context.WebApplicationContext
|
||||
import reactor.core.publisher.Sinks
|
||||
import reactor.test.StepVerifier
|
||||
import java.time.Instant
|
||||
|
||||
abstract class MockSink : Sinks.Many<Boolean>
|
||||
|
||||
@ -59,44 +78,50 @@ abstract class MockSink : Sinks.Many<Boolean>
|
||||
)
|
||||
@TestPropertySource(
|
||||
properties = [
|
||||
"app.pseudonymize.generator=BUILDIN",
|
||||
"app.security.admin-user=admin",
|
||||
"app.security.admin-password={noop}very-secret",
|
||||
"app.security.enable-tokens=true"
|
||||
"app.pseudonymize.generator=BUILDIN"
|
||||
]
|
||||
)
|
||||
@MockBean(name = "configsUpdateProducer", classes = [MockSink::class])
|
||||
@MockBean(
|
||||
@MockitoBean(name = "configsUpdateProducer", types = [MockSink::class])
|
||||
@MockitoBean(
|
||||
types = [
|
||||
Generator::class,
|
||||
MtbFileSender::class,
|
||||
ConnectionCheckService::class,
|
||||
RequestProcessor::class,
|
||||
TransformationService::class,
|
||||
TokenRepository::class,
|
||||
GPasConnectionCheckService::class,
|
||||
RestConnectionCheckService::class
|
||||
]
|
||||
)
|
||||
class ConfigControllerTest {
|
||||
|
||||
private lateinit var mockMvc: MockMvc
|
||||
private lateinit var webClient: WebClient
|
||||
|
||||
private lateinit var requestProcessor: RequestProcessor
|
||||
private lateinit var connectionCheckUpdateProducer: Sinks.Many<ConnectionCheckResult>
|
||||
|
||||
@BeforeEach
|
||||
fun setup(
|
||||
@Autowired mockMvc: MockMvc,
|
||||
@Autowired requestProcessor: RequestProcessor
|
||||
@Autowired requestProcessor: RequestProcessor,
|
||||
@Autowired connectionCheckUpdateProducer: Sinks.Many<ConnectionCheckResult>
|
||||
) {
|
||||
this.mockMvc = mockMvc
|
||||
this.webClient = MockMvcWebClientBuilder.mockMvcSetup(mockMvc).build()
|
||||
this.requestProcessor = requestProcessor
|
||||
this.connectionCheckUpdateProducer = connectionCheckUpdateProducer
|
||||
|
||||
webClient.options.isThrowExceptionOnScriptError = false
|
||||
}
|
||||
|
||||
@Test
|
||||
fun testShouldShowConfigPageIfLoggedIn() {
|
||||
fun testShouldRequestConfigPageIfLoggedIn() {
|
||||
mockMvc.get("/configs") {
|
||||
with(user("admin").roles("ADMIN"))
|
||||
accept(MediaType.TEXT_HTML)
|
||||
}.andExpect {
|
||||
status { isOk() }
|
||||
view { name("configs") }
|
||||
}
|
||||
}
|
||||
|
||||
@ -113,4 +138,232 @@ class ConfigControllerTest {
|
||||
}
|
||||
}
|
||||
|
||||
@Nested
|
||||
@TestPropertySource(
|
||||
properties = [
|
||||
"app.security.enable-tokens=true",
|
||||
"app.security.admin-user=admin"
|
||||
]
|
||||
)
|
||||
@MockitoBean(
|
||||
types = [
|
||||
TokenService::class
|
||||
]
|
||||
)
|
||||
inner class WithTokensEnabled {
|
||||
private lateinit var tokenService: TokenService
|
||||
|
||||
@BeforeEach
|
||||
fun setup(
|
||||
@Autowired tokenService: TokenService
|
||||
) {
|
||||
webClient.options.isThrowExceptionOnScriptError = false
|
||||
|
||||
this.tokenService = tokenService
|
||||
}
|
||||
|
||||
@Test
|
||||
fun testShouldSaveNewToken() {
|
||||
mockMvc.post("/configs/tokens") {
|
||||
with(user("admin").roles("ADMIN"))
|
||||
accept(MediaType.TEXT_HTML)
|
||||
contentType = MediaType.APPLICATION_FORM_URLENCODED
|
||||
content = "name=Testtoken"
|
||||
}.andExpect {
|
||||
status { is2xxSuccessful() }
|
||||
view { name("configs/tokens") }
|
||||
}
|
||||
|
||||
val captor = argumentCaptor<String>()
|
||||
verify(tokenService, times(1)).addToken(captor.capture())
|
||||
|
||||
assertThat(captor.firstValue).isEqualTo("Testtoken")
|
||||
}
|
||||
|
||||
@Test
|
||||
fun testShouldNotSaveTokenWithExstingName() {
|
||||
whenever(tokenService.addToken(anyString())).thenReturn(Result.failure(RuntimeException("Testfailure")))
|
||||
|
||||
mockMvc.post("/configs/tokens") {
|
||||
with(user("admin").roles("ADMIN"))
|
||||
accept(MediaType.TEXT_HTML)
|
||||
contentType = MediaType.APPLICATION_FORM_URLENCODED
|
||||
content = "name=Testtoken"
|
||||
}.andExpect {
|
||||
status { is2xxSuccessful() }
|
||||
view { name("configs/tokens") }
|
||||
}
|
||||
|
||||
val captor = argumentCaptor<String>()
|
||||
verify(tokenService, times(1)).addToken(captor.capture())
|
||||
|
||||
assertThat(captor.firstValue).isEqualTo("Testtoken")
|
||||
}
|
||||
|
||||
@Test
|
||||
fun testShouldDeleteToken() {
|
||||
mockMvc.delete("/configs/tokens/42") {
|
||||
with(user("admin").roles("ADMIN"))
|
||||
accept(MediaType.TEXT_HTML)
|
||||
}.andExpect {
|
||||
status { is2xxSuccessful() }
|
||||
view { name("configs/tokens") }
|
||||
}
|
||||
|
||||
val captor = argumentCaptor<Long>()
|
||||
verify(tokenService, times(1)).deleteToken(captor.capture())
|
||||
|
||||
assertThat(captor.firstValue).isEqualTo(42)
|
||||
}
|
||||
|
||||
@Test
|
||||
@WithMockUser(username = "admin", roles = ["ADMIN"])
|
||||
fun testShouldRenderConfigPageWithTokens() {
|
||||
val page = webClient.getPage<HtmlPage>("http://localhost/configs")
|
||||
assertThat(
|
||||
page.getElementById("tokens")
|
||||
).isNotNull
|
||||
}
|
||||
}
|
||||
|
||||
@Nested
|
||||
@TestPropertySource(
|
||||
properties = [
|
||||
"app.security.enable-tokens=false"
|
||||
]
|
||||
)
|
||||
inner class WithTokensDisabled {
|
||||
@BeforeEach
|
||||
fun setup() {
|
||||
webClient.options.isThrowExceptionOnScriptError = false
|
||||
}
|
||||
|
||||
@Test
|
||||
@WithMockUser(username = "admin", roles = ["ADMIN"])
|
||||
fun testShouldRenderConfigPageWithoutTokens() {
|
||||
val page = webClient.getPage<HtmlPage>("http://localhost/configs")
|
||||
assertThat(
|
||||
page.getElementById("tokens")
|
||||
).isNull()
|
||||
}
|
||||
}
|
||||
|
||||
@Nested
|
||||
@TestPropertySource(
|
||||
properties = [
|
||||
"app.security.enable-tokens=false",
|
||||
"app.security.admin-user=admin",
|
||||
"app.security.admin-password={noop}very-secret"
|
||||
]
|
||||
)
|
||||
@MockitoBean(
|
||||
types = [
|
||||
UserRoleService::class
|
||||
]
|
||||
)
|
||||
inner class WithUserRolesEnabled {
|
||||
private lateinit var userRoleService: UserRoleService
|
||||
|
||||
@BeforeEach
|
||||
fun setup(
|
||||
@Autowired userRoleService: UserRoleService
|
||||
) {
|
||||
webClient.options.isThrowExceptionOnScriptError = false
|
||||
|
||||
this.userRoleService = userRoleService
|
||||
}
|
||||
|
||||
@Test
|
||||
fun testShouldDeleteUserRole() {
|
||||
mockMvc.delete("/configs/userroles/42") {
|
||||
with(user("admin").roles("ADMIN"))
|
||||
accept(MediaType.TEXT_HTML)
|
||||
}.andExpect {
|
||||
status { is2xxSuccessful() }
|
||||
view { name("configs/userroles") }
|
||||
}
|
||||
|
||||
val captor = argumentCaptor<Long>()
|
||||
verify(userRoleService, times(1)).deleteUserRole(captor.capture())
|
||||
|
||||
assertThat(captor.firstValue).isEqualTo(42)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun testShouldUpdateUserRole() {
|
||||
mockMvc.put("/configs/userroles/42") {
|
||||
with(user("admin").roles("ADMIN"))
|
||||
accept(MediaType.TEXT_HTML)
|
||||
contentType = MediaType.APPLICATION_FORM_URLENCODED
|
||||
content = "role=ADMIN"
|
||||
}.andExpect {
|
||||
status { is2xxSuccessful() }
|
||||
view { name("configs/userroles") }
|
||||
}
|
||||
|
||||
val idCaptor = argumentCaptor<Long>()
|
||||
val roleCaptor = argumentCaptor<Role>()
|
||||
verify(userRoleService, times(1)).updateUserRole(idCaptor.capture(), roleCaptor.capture())
|
||||
|
||||
assertThat(idCaptor.firstValue).isEqualTo(42)
|
||||
assertThat(roleCaptor.firstValue).isEqualTo(Role.ADMIN)
|
||||
}
|
||||
|
||||
@Test
|
||||
@WithMockUser(username = "admin", roles = ["ADMIN"])
|
||||
fun testShouldRenderConfigPageWithUserRoles() {
|
||||
val page = webClient.getPage<HtmlPage>("http://localhost/configs")
|
||||
assertThat(
|
||||
page.getElementById("userroles")
|
||||
).isNotNull
|
||||
}
|
||||
}
|
||||
|
||||
@Nested
|
||||
inner class WithUserRolesDisabled {
|
||||
@BeforeEach
|
||||
fun setup() {
|
||||
webClient.options.isThrowExceptionOnScriptError = false
|
||||
}
|
||||
|
||||
@Test
|
||||
fun testShouldRenderConfigPageWithoutUserRoles() {
|
||||
val page = webClient.getPage<HtmlPage>("http://localhost/configs")
|
||||
assertThat(
|
||||
page.getElementById("userroles")
|
||||
).isNull()
|
||||
}
|
||||
}
|
||||
|
||||
@Nested
|
||||
inner class SseTest {
|
||||
private lateinit var webClient: WebTestClient
|
||||
|
||||
@BeforeEach
|
||||
fun setup(
|
||||
applicationContext: WebApplicationContext,
|
||||
) {
|
||||
this.webClient = MockMvcWebTestClient
|
||||
.bindToApplicationContext(applicationContext).build()
|
||||
}
|
||||
|
||||
@Test
|
||||
fun testShouldRequestSSE() {
|
||||
val expectedEvent = ConnectionCheckResult.GPasConnectionCheckResult(true, Instant.now(), Instant.now())
|
||||
|
||||
connectionCheckUpdateProducer.tryEmitNext(expectedEvent)
|
||||
connectionCheckUpdateProducer.emitComplete { _, _ -> true }
|
||||
|
||||
val result = webClient.get().uri("http://localhost/configs/events").accept(TEXT_EVENT_STREAM).exchange()
|
||||
.expectStatus().isOk()
|
||||
.expectHeader().contentType(TEXT_EVENT_STREAM)
|
||||
.returnResult(ConnectionCheckResult.GPasConnectionCheckResult::class.java)
|
||||
|
||||
StepVerifier.create(result.responseBody)
|
||||
.expectNext(expectedEvent)
|
||||
.expectComplete()
|
||||
.verify()
|
||||
}
|
||||
}
|
||||
|
||||
}
|
||||
|
@ -0,0 +1,287 @@
|
||||
/*
|
||||
* This file is part of ETL-Processor
|
||||
*
|
||||
* Copyright (c) 2024 Comprehensive Cancer Center Mainfranken, Datenintegrationszentrum Philipps-Universität Marburg and Contributors
|
||||
*
|
||||
* This program is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
* by the Free Software Foundation, either version 3 of the License, or
|
||||
* (at your option) any later version.
|
||||
*
|
||||
* This program is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
* GNU Affero General Public License for more details.
|
||||
*
|
||||
* You should have received a copy of the GNU Affero General Public License
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
*/
|
||||
|
||||
package dev.dnpm.etl.processor.web
|
||||
|
||||
import dev.dnpm.etl.processor.*
|
||||
import dev.dnpm.etl.processor.config.AppConfiguration
|
||||
import dev.dnpm.etl.processor.config.AppSecurityConfiguration
|
||||
import dev.dnpm.etl.processor.monitoring.Report
|
||||
import dev.dnpm.etl.processor.monitoring.Request
|
||||
import dev.dnpm.etl.processor.monitoring.RequestStatus
|
||||
import dev.dnpm.etl.processor.monitoring.RequestType
|
||||
import dev.dnpm.etl.processor.services.RequestService
|
||||
import org.assertj.core.api.Assertions.assertThat
|
||||
import org.htmlunit.WebClient
|
||||
import org.htmlunit.html.HtmlPage
|
||||
import org.junit.jupiter.api.BeforeEach
|
||||
import org.junit.jupiter.api.Nested
|
||||
import org.junit.jupiter.api.Test
|
||||
import org.junit.jupiter.api.assertThrows
|
||||
import org.junit.jupiter.api.extension.ExtendWith
|
||||
import org.mockito.junit.jupiter.MockitoExtension
|
||||
import org.mockito.kotlin.any
|
||||
import org.mockito.kotlin.whenever
|
||||
import org.springframework.beans.factory.annotation.Autowired
|
||||
import org.springframework.boot.test.autoconfigure.web.servlet.WebMvcTest
|
||||
import org.springframework.data.domain.Page
|
||||
import org.springframework.data.domain.PageImpl
|
||||
import org.springframework.data.domain.Pageable
|
||||
import org.springframework.security.test.context.support.WithMockUser
|
||||
import org.springframework.test.context.ContextConfiguration
|
||||
import org.springframework.test.context.TestPropertySource
|
||||
import org.springframework.test.context.bean.override.mockito.MockitoBean
|
||||
import org.springframework.test.context.junit.jupiter.SpringExtension
|
||||
import org.springframework.test.web.servlet.MockMvc
|
||||
import org.springframework.test.web.servlet.get
|
||||
import org.springframework.test.web.servlet.htmlunit.MockMvcWebClientBuilder
|
||||
import java.io.IOException
|
||||
import java.time.Instant
|
||||
import java.util.*
|
||||
|
||||
@WebMvcTest(controllers = [HomeController::class])
|
||||
@ExtendWith(value = [MockitoExtension::class, SpringExtension::class])
|
||||
@ContextConfiguration(
|
||||
classes = [
|
||||
HomeController::class,
|
||||
AppConfiguration::class,
|
||||
AppSecurityConfiguration::class
|
||||
]
|
||||
)
|
||||
@TestPropertySource(
|
||||
properties = [
|
||||
"app.pseudonymize.generator=BUILDIN",
|
||||
"app.security.admin-user=admin",
|
||||
"app.security.admin-password={noop}very-secret"
|
||||
]
|
||||
)
|
||||
@MockitoBean(
|
||||
types = [RequestService::class]
|
||||
)
|
||||
class HomeControllerTest {
|
||||
|
||||
private lateinit var mockMvc: MockMvc
|
||||
private lateinit var webClient: WebClient
|
||||
|
||||
@BeforeEach
|
||||
fun setup(
|
||||
@Autowired mockMvc: MockMvc,
|
||||
@Autowired requestService: RequestService
|
||||
) {
|
||||
this.mockMvc = mockMvc
|
||||
this.webClient = MockMvcWebClientBuilder.mockMvcSetup(mockMvc).build()
|
||||
|
||||
whenever(requestService.findAll(any<Pageable>())).thenReturn(Page.empty())
|
||||
}
|
||||
|
||||
@Test
|
||||
fun testShouldRequestHomePage() {
|
||||
mockMvc.get("/").andExpect {
|
||||
status { isOk() }
|
||||
view { name("index") }
|
||||
}
|
||||
}
|
||||
|
||||
@Nested
|
||||
inner class WithRequests {
|
||||
|
||||
private lateinit var requestService: RequestService
|
||||
|
||||
@BeforeEach
|
||||
fun setup(
|
||||
@Autowired requestService: RequestService
|
||||
) {
|
||||
this.requestService = requestService
|
||||
}
|
||||
|
||||
@Test
|
||||
fun testShouldShowHomePage() {
|
||||
whenever(requestService.findAll(any<Pageable>())).thenReturn(
|
||||
PageImpl(
|
||||
listOf(
|
||||
Request(
|
||||
2L,
|
||||
randomRequestId(),
|
||||
PatientPseudonym("PSEUDO1"),
|
||||
PatientId("PATIENT1"),
|
||||
Fingerprint("ashdkasdh"),
|
||||
RequestType.MTB_FILE,
|
||||
RequestStatus.SUCCESS
|
||||
),
|
||||
Request(
|
||||
1L,
|
||||
randomRequestId(),
|
||||
PatientPseudonym("PSEUDO1"),
|
||||
PatientId("PATIENT1"),
|
||||
Fingerprint("asdasdasd"),
|
||||
RequestType.MTB_FILE,
|
||||
RequestStatus.ERROR
|
||||
)
|
||||
)
|
||||
)
|
||||
)
|
||||
|
||||
val page = webClient.getPage<HtmlPage>("http://localhost/")
|
||||
assertThat(page.querySelectorAll("tbody tr")).hasSize(2)
|
||||
assertThat(page.querySelectorAll("div.notification.info")).isEmpty()
|
||||
}
|
||||
|
||||
@Test
|
||||
@WithMockUser(username = "admin", roles = ["ADMIN"])
|
||||
fun testShouldShowRequestDetails() {
|
||||
val requestId = randomRequestId()
|
||||
|
||||
whenever(requestService.findByUuid(anyValueClass())).thenReturn(
|
||||
Optional.of(
|
||||
Request(
|
||||
2L,
|
||||
requestId,
|
||||
PatientPseudonym("PSEUDO1"),
|
||||
PatientId("PATIENT1"),
|
||||
Fingerprint("ashdkasdh"),
|
||||
RequestType.MTB_FILE,
|
||||
RequestStatus.SUCCESS,
|
||||
Instant.now(),
|
||||
Report("Test")
|
||||
)
|
||||
)
|
||||
)
|
||||
|
||||
val page = webClient.getPage<HtmlPage>("http://localhost/report/${requestId.value}")
|
||||
assertThat(page.querySelectorAll("tbody tr")).hasSize(1)
|
||||
assertThat(page.querySelectorAll("div.notification.info")).isEmpty()
|
||||
}
|
||||
|
||||
@Test
|
||||
@WithMockUser(username = "admin", roles = ["ADMIN"])
|
||||
fun testShouldShowPatientDetails() {
|
||||
whenever(requestService.findRequestByPatientId(anyValueClass(), any<Pageable>())).thenReturn(
|
||||
PageImpl(
|
||||
listOf(
|
||||
Request(
|
||||
2L,
|
||||
randomRequestId(),
|
||||
PatientPseudonym("PSEUDO1"),
|
||||
PatientId("PATIENT1"),
|
||||
Fingerprint("ashdkasdh"),
|
||||
RequestType.MTB_FILE,
|
||||
RequestStatus.SUCCESS
|
||||
),
|
||||
Request(
|
||||
1L,
|
||||
randomRequestId(),
|
||||
PatientPseudonym("PSEUDO1"),
|
||||
PatientId("PATIENT1"),
|
||||
Fingerprint("asdasdasd"),
|
||||
RequestType.MTB_FILE,
|
||||
RequestStatus.ERROR
|
||||
)
|
||||
)
|
||||
)
|
||||
)
|
||||
|
||||
val page = webClient.getPage<HtmlPage>("http://localhost/patient/PSEUDO1")
|
||||
assertThat(page.querySelectorAll("tbody tr")).hasSize(2)
|
||||
assertThat(page.querySelectorAll("div.notification.info")).isEmpty()
|
||||
}
|
||||
|
||||
@Test
|
||||
@WithMockUser(username = "admin", roles = ["ADMIN"])
|
||||
fun testShouldShowPatientPseudonym() {
|
||||
whenever(requestService.findRequestByPatientId(anyValueClass(), any<Pageable>())).thenReturn(
|
||||
PageImpl(
|
||||
listOf(
|
||||
Request(
|
||||
2L,
|
||||
randomRequestId(),
|
||||
PatientPseudonym("PSEUDO1"),
|
||||
PatientId("PATIENT1"),
|
||||
Fingerprint("ashdkasdh"),
|
||||
RequestType.MTB_FILE,
|
||||
RequestStatus.SUCCESS
|
||||
),
|
||||
Request(
|
||||
1L,
|
||||
randomRequestId(),
|
||||
PatientPseudonym("PSEUDO1"),
|
||||
PatientId("PATIENT1"),
|
||||
Fingerprint("asdasdasd"),
|
||||
RequestType.MTB_FILE,
|
||||
RequestStatus.ERROR
|
||||
)
|
||||
)
|
||||
)
|
||||
)
|
||||
|
||||
val page = webClient.getPage<HtmlPage>("http://localhost/patient/PSEUDO1")
|
||||
assertThat(page.querySelectorAll("h2 > span")).hasSize(1)
|
||||
assertThat(page.querySelectorAll("h2 > span").first().textContent).isEqualTo("PSEUDO1")
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
@Nested
|
||||
inner class WithoutRequests {
|
||||
|
||||
private lateinit var requestService: RequestService
|
||||
|
||||
@BeforeEach
|
||||
fun setup(
|
||||
@Autowired requestService: RequestService
|
||||
) {
|
||||
this.requestService = requestService
|
||||
|
||||
whenever(requestService.findAll(any<Pageable>())).thenReturn(Page.empty())
|
||||
}
|
||||
|
||||
@Test
|
||||
fun testShouldShowHomePage() {
|
||||
val page = webClient.getPage<HtmlPage>("http://localhost/")
|
||||
assertThat(page.querySelectorAll("tbody tr")).isEmpty()
|
||||
assertThat(page.querySelectorAll("div.notification.info")).hasSize(1)
|
||||
}
|
||||
|
||||
@Test
|
||||
@WithMockUser(username = "admin", roles = ["ADMIN"])
|
||||
fun testShouldThrowNotFoundExceptionForUnknownReport() {
|
||||
val requestId = randomRequestId()
|
||||
|
||||
whenever(requestService.findByUuid(anyValueClass())).thenReturn(
|
||||
Optional.empty()
|
||||
)
|
||||
|
||||
assertThrows<IOException> {
|
||||
webClient.getPage<HtmlPage>("http://localhost/report/${requestId.value}")
|
||||
}.also {
|
||||
assertThat(it).hasRootCauseInstanceOf(NotFoundException::class.java)
|
||||
}
|
||||
}
|
||||
|
||||
@Test
|
||||
@WithMockUser(username = "admin", roles = ["ADMIN"])
|
||||
fun testShouldShowEmptyPatientDetails() {
|
||||
whenever(requestService.findRequestByPatientId(anyValueClass(), any<Pageable>())).thenReturn(Page.empty())
|
||||
|
||||
val page = webClient.getPage<HtmlPage>("http://localhost/patient/PSEUDO1")
|
||||
assertThat(page.querySelectorAll("tbody tr")).isEmpty()
|
||||
assertThat(page.querySelectorAll("div.notification.info")).hasSize(1)
|
||||
}
|
||||
}
|
||||
|
||||
}
|
@ -0,0 +1,88 @@
|
||||
/*
|
||||
* This file is part of ETL-Processor
|
||||
*
|
||||
* Copyright (c) 2024 Comprehensive Cancer Center Mainfranken, Datenintegrationszentrum Philipps-Universität Marburg and Contributors
|
||||
*
|
||||
* This program is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
* by the Free Software Foundation, either version 3 of the License, or
|
||||
* (at your option) any later version.
|
||||
*
|
||||
* This program is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
* GNU Affero General Public License for more details.
|
||||
*
|
||||
* You should have received a copy of the GNU Affero General Public License
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
*/
|
||||
|
||||
package dev.dnpm.etl.processor.web
|
||||
|
||||
import dev.dnpm.etl.processor.config.AppConfiguration
|
||||
import dev.dnpm.etl.processor.config.AppSecurityConfiguration
|
||||
import dev.dnpm.etl.processor.security.TokenService
|
||||
import org.assertj.core.api.Assertions.assertThat
|
||||
import org.htmlunit.WebClient
|
||||
import org.htmlunit.html.HtmlPage
|
||||
import org.junit.jupiter.api.BeforeEach
|
||||
import org.junit.jupiter.api.Test
|
||||
import org.junit.jupiter.api.extension.ExtendWith
|
||||
import org.mockito.junit.jupiter.MockitoExtension
|
||||
import org.springframework.beans.factory.annotation.Autowired
|
||||
import org.springframework.boot.test.autoconfigure.web.servlet.WebMvcTest
|
||||
import org.springframework.test.context.ContextConfiguration
|
||||
import org.springframework.test.context.TestPropertySource
|
||||
import org.springframework.test.context.bean.override.mockito.MockitoBean
|
||||
import org.springframework.test.context.junit.jupiter.SpringExtension
|
||||
import org.springframework.test.web.servlet.MockMvc
|
||||
import org.springframework.test.web.servlet.get
|
||||
import org.springframework.test.web.servlet.htmlunit.MockMvcWebClientBuilder
|
||||
|
||||
@WebMvcTest(controllers = [LoginController::class])
|
||||
@ExtendWith(value = [MockitoExtension::class, SpringExtension::class])
|
||||
@ContextConfiguration(
|
||||
classes = [
|
||||
LoginController::class,
|
||||
AppConfiguration::class,
|
||||
AppSecurityConfiguration::class
|
||||
]
|
||||
)
|
||||
@TestPropertySource(
|
||||
properties = [
|
||||
"app.pseudonymize.generator=BUILDIN",
|
||||
"app.security.admin-user=admin",
|
||||
"app.security.admin-password={noop}very-secret",
|
||||
"app.security.enable-tokens=true"
|
||||
]
|
||||
)
|
||||
@MockitoBean(
|
||||
types = [TokenService::class]
|
||||
)
|
||||
class LoginControllerTest {
|
||||
|
||||
private lateinit var mockMvc: MockMvc
|
||||
private lateinit var webClient: WebClient
|
||||
|
||||
@BeforeEach
|
||||
fun setup(@Autowired mockMvc: MockMvc) {
|
||||
this.mockMvc = mockMvc
|
||||
this.webClient = MockMvcWebClientBuilder.mockMvcSetup(mockMvc).build()
|
||||
}
|
||||
|
||||
@Test
|
||||
fun testShouldRequestLoginPage() {
|
||||
mockMvc.get("/login").andExpect {
|
||||
status { isOk() }
|
||||
view { name("login") }
|
||||
}
|
||||
}
|
||||
|
||||
@Test
|
||||
fun testShouldShowLoginForm() {
|
||||
val page = webClient.getPage<HtmlPage>("http://localhost/login")
|
||||
assertThat(
|
||||
page.getElementsByTagName("main").first().firstElementChild.getAttribute("class")
|
||||
).isEqualTo("login-form")
|
||||
}
|
||||
}
|
@ -0,0 +1,73 @@
|
||||
/*
|
||||
* This file is part of ETL-Processor
|
||||
*
|
||||
* Copyright (c) 2024 Comprehensive Cancer Center Mainfranken, Datenintegrationszentrum Philipps-Universität Marburg and Contributors
|
||||
*
|
||||
* This program is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
* by the Free Software Foundation, either version 3 of the License, or
|
||||
* (at your option) any later version.
|
||||
*
|
||||
* This program is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
* GNU Affero General Public License for more details.
|
||||
*
|
||||
* You should have received a copy of the GNU Affero General Public License
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
*/
|
||||
|
||||
package dev.dnpm.etl.processor.web
|
||||
|
||||
import dev.dnpm.etl.processor.config.AppConfiguration
|
||||
import dev.dnpm.etl.processor.config.AppSecurityConfiguration
|
||||
import org.htmlunit.WebClient
|
||||
import org.junit.jupiter.api.BeforeEach
|
||||
import org.junit.jupiter.api.Test
|
||||
import org.junit.jupiter.api.extension.ExtendWith
|
||||
import org.mockito.junit.jupiter.MockitoExtension
|
||||
import org.springframework.beans.factory.annotation.Autowired
|
||||
import org.springframework.boot.test.autoconfigure.web.servlet.WebMvcTest
|
||||
import org.springframework.test.context.ContextConfiguration
|
||||
import org.springframework.test.context.TestPropertySource
|
||||
import org.springframework.test.context.junit.jupiter.SpringExtension
|
||||
import org.springframework.test.web.servlet.MockMvc
|
||||
import org.springframework.test.web.servlet.get
|
||||
import org.springframework.test.web.servlet.htmlunit.MockMvcWebClientBuilder
|
||||
|
||||
@WebMvcTest(controllers = [StatisticsController::class])
|
||||
@ExtendWith(value = [MockitoExtension::class, SpringExtension::class])
|
||||
@ContextConfiguration(
|
||||
classes = [
|
||||
StatisticsController::class,
|
||||
AppConfiguration::class,
|
||||
AppSecurityConfiguration::class
|
||||
]
|
||||
)
|
||||
@TestPropertySource(
|
||||
properties = [
|
||||
"app.pseudonymize.generator=BUILDIN",
|
||||
"app.security.admin-user=admin",
|
||||
"app.security.admin-password={noop}very-secret"
|
||||
]
|
||||
)
|
||||
class StatisticsControllerTest {
|
||||
|
||||
private lateinit var mockMvc: MockMvc
|
||||
private lateinit var webClient: WebClient
|
||||
|
||||
@BeforeEach
|
||||
fun setup(@Autowired mockMvc: MockMvc) {
|
||||
this.mockMvc = mockMvc
|
||||
this.webClient = MockMvcWebClientBuilder.mockMvcSetup(mockMvc).build()
|
||||
}
|
||||
|
||||
@Test
|
||||
fun testShouldRequestLoginPage() {
|
||||
mockMvc.get("/statistics").andExpect {
|
||||
status { isOk() }
|
||||
view { name("statistics") }
|
||||
}
|
||||
}
|
||||
|
||||
}
|
@ -0,0 +1,314 @@
|
||||
/*
|
||||
* This file is part of ETL-Processor
|
||||
*
|
||||
* Copyright (c) 2024 Comprehensive Cancer Center Mainfranken, Datenintegrationszentrum Philipps-Universität Marburg and Contributors
|
||||
*
|
||||
* This program is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
* by the Free Software Foundation, either version 3 of the License, or
|
||||
* (at your option) any later version.
|
||||
*
|
||||
* This program is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
* GNU Affero General Public License for more details.
|
||||
*
|
||||
* You should have received a copy of the GNU Affero General Public License
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
*/
|
||||
|
||||
package dev.dnpm.etl.processor.web
|
||||
|
||||
import dev.dnpm.etl.processor.Fingerprint
|
||||
import dev.dnpm.etl.processor.PatientId
|
||||
import dev.dnpm.etl.processor.PatientPseudonym
|
||||
import dev.dnpm.etl.processor.config.AppConfiguration
|
||||
import dev.dnpm.etl.processor.config.AppSecurityConfiguration
|
||||
import dev.dnpm.etl.processor.monitoring.CountedState
|
||||
import dev.dnpm.etl.processor.monitoring.Request
|
||||
import dev.dnpm.etl.processor.monitoring.RequestStatus
|
||||
import dev.dnpm.etl.processor.monitoring.RequestType
|
||||
import dev.dnpm.etl.processor.randomRequestId
|
||||
import dev.dnpm.etl.processor.services.RequestService
|
||||
import org.hamcrest.Matchers.equalTo
|
||||
import org.hamcrest.Matchers.hasSize
|
||||
import org.junit.jupiter.api.BeforeEach
|
||||
import org.junit.jupiter.api.Nested
|
||||
import org.junit.jupiter.api.Test
|
||||
import org.junit.jupiter.api.extension.ExtendWith
|
||||
import org.mockito.junit.jupiter.MockitoExtension
|
||||
import org.mockito.kotlin.doAnswer
|
||||
import org.mockito.kotlin.whenever
|
||||
import org.springframework.beans.factory.annotation.Autowired
|
||||
import org.springframework.boot.test.autoconfigure.web.servlet.WebMvcTest
|
||||
import org.springframework.http.MediaType.TEXT_EVENT_STREAM
|
||||
import org.springframework.test.context.ContextConfiguration
|
||||
import org.springframework.test.context.TestPropertySource
|
||||
import org.springframework.test.context.bean.override.mockito.MockitoBean
|
||||
import org.springframework.test.context.junit.jupiter.SpringExtension
|
||||
import org.springframework.test.web.reactive.server.WebTestClient
|
||||
import org.springframework.test.web.servlet.MockMvc
|
||||
import org.springframework.test.web.servlet.client.MockMvcWebTestClient
|
||||
import org.springframework.test.web.servlet.get
|
||||
import org.springframework.web.context.WebApplicationContext
|
||||
import reactor.core.publisher.Sinks
|
||||
import reactor.test.StepVerifier
|
||||
import java.time.Instant
|
||||
import java.time.ZoneId
|
||||
import java.time.temporal.ChronoUnit
|
||||
|
||||
|
||||
@WebMvcTest(controllers = [StatisticsRestController::class])
|
||||
@ExtendWith(value = [MockitoExtension::class, SpringExtension::class])
|
||||
@ContextConfiguration(
|
||||
classes = [
|
||||
StatisticsRestController::class,
|
||||
AppConfiguration::class,
|
||||
AppSecurityConfiguration::class
|
||||
]
|
||||
)
|
||||
@TestPropertySource(
|
||||
properties = [
|
||||
"app.pseudonymize.generator=BUILDIN",
|
||||
"app.security.admin-user=admin",
|
||||
"app.security.admin-password={noop}very-secret"
|
||||
]
|
||||
)
|
||||
@MockitoBean(
|
||||
types = [RequestService::class]
|
||||
)
|
||||
class StatisticsRestControllerTest {
|
||||
|
||||
private lateinit var mockMvc: MockMvc
|
||||
|
||||
private lateinit var statisticsUpdateProducer: Sinks.Many<Any>
|
||||
private lateinit var requestService: RequestService
|
||||
|
||||
@BeforeEach
|
||||
fun setup(
|
||||
@Autowired mockMvc: MockMvc,
|
||||
@Autowired statisticsUpdateProducer: Sinks.Many<Any>,
|
||||
@Autowired requestService: RequestService
|
||||
) {
|
||||
this.mockMvc = mockMvc
|
||||
this.statisticsUpdateProducer = statisticsUpdateProducer
|
||||
this.requestService = requestService
|
||||
}
|
||||
|
||||
@Nested
|
||||
inner class RequestStatesTest {
|
||||
@Test
|
||||
fun testShouldRequestStatesForMtbFiles() {
|
||||
doAnswer { _ ->
|
||||
listOf(
|
||||
CountedState(42, RequestStatus.WARNING),
|
||||
CountedState(1, RequestStatus.UNKNOWN)
|
||||
)
|
||||
}.whenever(requestService).countStates()
|
||||
|
||||
mockMvc.get("/statistics/requeststates").andExpect {
|
||||
status { isOk() }.also {
|
||||
jsonPath("$", hasSize<Int>(2))
|
||||
jsonPath("$[0].name", equalTo(RequestStatus.WARNING.name))
|
||||
jsonPath("$[0].value", equalTo(42))
|
||||
jsonPath("$[1].name", equalTo(RequestStatus.UNKNOWN.name))
|
||||
jsonPath("$[1].value", equalTo(1))
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@Test
|
||||
fun testShouldRequestStatesForDeletes() {
|
||||
doAnswer { _ ->
|
||||
listOf(
|
||||
CountedState(42, RequestStatus.SUCCESS),
|
||||
CountedState(1, RequestStatus.ERROR)
|
||||
)
|
||||
}.whenever(requestService).countDeleteStates()
|
||||
|
||||
mockMvc.get("/statistics/requeststates?delete=true").andExpect {
|
||||
status { isOk() }.also {
|
||||
jsonPath("$", hasSize<Int>(2))
|
||||
jsonPath("$[0].name", equalTo(RequestStatus.SUCCESS.name))
|
||||
jsonPath("$[0].value", equalTo(42))
|
||||
jsonPath("$[1].name", equalTo(RequestStatus.ERROR.name))
|
||||
jsonPath("$[1].value", equalTo(1))
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@Nested
|
||||
inner class PatientRequestStatesTest {
|
||||
@Test
|
||||
fun testShouldRequestPatientStatesForMtbFiles() {
|
||||
doAnswer { _ ->
|
||||
listOf(
|
||||
CountedState(42, RequestStatus.WARNING),
|
||||
CountedState(1, RequestStatus.UNKNOWN)
|
||||
)
|
||||
}.whenever(requestService).findPatientUniqueStates()
|
||||
|
||||
mockMvc.get("/statistics/requestpatientstates").andExpect {
|
||||
status { isOk() }.also {
|
||||
jsonPath("$", hasSize<Int>(2))
|
||||
jsonPath("$[0].name", equalTo(RequestStatus.WARNING.name))
|
||||
jsonPath("$[0].value", equalTo(42))
|
||||
jsonPath("$[1].name", equalTo(RequestStatus.UNKNOWN.name))
|
||||
jsonPath("$[1].value", equalTo(1))
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@Test
|
||||
fun testShouldRequestPatientStatesForDeletes() {
|
||||
doAnswer { _ ->
|
||||
listOf(
|
||||
CountedState(42, RequestStatus.SUCCESS),
|
||||
CountedState(1, RequestStatus.ERROR)
|
||||
)
|
||||
}.whenever(requestService).findPatientUniqueDeleteStates()
|
||||
|
||||
mockMvc.get("/statistics/requestpatientstates?delete=true").andExpect {
|
||||
status { isOk() }.also {
|
||||
jsonPath("$", hasSize<Int>(2))
|
||||
jsonPath("$[0].name", equalTo(RequestStatus.SUCCESS.name))
|
||||
jsonPath("$[0].value", equalTo(42))
|
||||
jsonPath("$[1].name", equalTo(RequestStatus.ERROR.name))
|
||||
jsonPath("$[1].value", equalTo(1))
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@Nested
|
||||
inner class LastMonthStatesTest {
|
||||
|
||||
@BeforeEach
|
||||
fun setup() {
|
||||
val zoneId = ZoneId.of("Europe/Berlin")
|
||||
doAnswer { _ ->
|
||||
listOf(
|
||||
Request(
|
||||
1,
|
||||
randomRequestId(),
|
||||
PatientPseudonym("TEST_12345678901"),
|
||||
PatientId("P1"),
|
||||
Fingerprint("0123456789abcdef1"),
|
||||
RequestType.MTB_FILE,
|
||||
RequestStatus.SUCCESS,
|
||||
Instant.now().atZone(zoneId).truncatedTo(ChronoUnit.DAYS).minus(2, ChronoUnit.DAYS).toInstant()
|
||||
),
|
||||
Request(
|
||||
2,
|
||||
randomRequestId(),
|
||||
PatientPseudonym("TEST_12345678902"),
|
||||
PatientId("P2"),
|
||||
Fingerprint("0123456789abcdef2"),
|
||||
RequestType.MTB_FILE,
|
||||
RequestStatus.WARNING,
|
||||
Instant.now().atZone(zoneId).truncatedTo(ChronoUnit.DAYS).minus(2, ChronoUnit.DAYS).toInstant()
|
||||
),
|
||||
Request(
|
||||
3,
|
||||
randomRequestId(),
|
||||
PatientPseudonym("TEST_12345678901"),
|
||||
PatientId("P2"),
|
||||
Fingerprint("0123456789abcdee1"),
|
||||
RequestType.DELETE,
|
||||
RequestStatus.ERROR,
|
||||
Instant.now().atZone(zoneId).truncatedTo(ChronoUnit.DAYS).minus(1, ChronoUnit.DAYS).toInstant()
|
||||
),
|
||||
Request(
|
||||
4,
|
||||
randomRequestId(),
|
||||
PatientPseudonym("TEST_12345678902"),
|
||||
PatientId("P2"),
|
||||
Fingerprint("0123456789abcdef2"),
|
||||
RequestType.MTB_FILE,
|
||||
RequestStatus.DUPLICATION,
|
||||
Instant.now().atZone(zoneId).truncatedTo(ChronoUnit.DAYS).minus(1, ChronoUnit.DAYS).toInstant()
|
||||
),
|
||||
Request(
|
||||
5,
|
||||
randomRequestId(),
|
||||
PatientPseudonym("TEST_12345678902"),
|
||||
PatientId("P2"),
|
||||
Fingerprint("0123456789abcdef2"),
|
||||
RequestType.DELETE,
|
||||
RequestStatus.UNKNOWN,
|
||||
Instant.now().atZone(zoneId).truncatedTo(ChronoUnit.DAYS).toInstant()
|
||||
),
|
||||
)
|
||||
}.whenever(requestService).findAll()
|
||||
}
|
||||
|
||||
@Test
|
||||
fun testShouldRequestLastMonthForMtbFiles() {
|
||||
mockMvc.get("/statistics/requestslastmonth").andExpect {
|
||||
status { isOk() }.also {
|
||||
jsonPath("$", hasSize<Int>(31))
|
||||
}.also {
|
||||
jsonPath("$[28].nameValues.error", equalTo(0))
|
||||
jsonPath("$[28].nameValues.warning", equalTo(1))
|
||||
jsonPath("$[28].nameValues.success", equalTo(1))
|
||||
jsonPath("$[28].nameValues.duplication", equalTo(0))
|
||||
jsonPath("$[28].nameValues.unknown", equalTo(0))
|
||||
jsonPath("$[29].nameValues.error", equalTo(0))
|
||||
jsonPath("$[29].nameValues.warning", equalTo(0))
|
||||
jsonPath("$[29].nameValues.success", equalTo(0))
|
||||
jsonPath("$[29].nameValues.duplication", equalTo(1))
|
||||
jsonPath("$[29].nameValues.unknown", equalTo(0))
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@Test
|
||||
fun testShouldRequestLastMonthForDeletes() {
|
||||
mockMvc.get("/statistics/requestslastmonth?delete=true").andExpect {
|
||||
status { isOk() }.also {
|
||||
jsonPath("$", hasSize<Int>(31))
|
||||
}.also {
|
||||
jsonPath("$[29].nameValues.error", equalTo(1))
|
||||
jsonPath("$[29].nameValues.warning", equalTo(0))
|
||||
jsonPath("$[29].nameValues.success", equalTo(0))
|
||||
jsonPath("$[29].nameValues.duplication", equalTo(0))
|
||||
jsonPath("$[29].nameValues.unknown", equalTo(0))
|
||||
jsonPath("$[30].nameValues.error", equalTo(0))
|
||||
jsonPath("$[30].nameValues.warning", equalTo(0))
|
||||
jsonPath("$[30].nameValues.success", equalTo(0))
|
||||
jsonPath("$[30].nameValues.duplication", equalTo(0))
|
||||
jsonPath("$[30].nameValues.unknown", equalTo(1))
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@Nested
|
||||
inner class SseTest {
|
||||
private lateinit var webClient: WebTestClient
|
||||
|
||||
@BeforeEach
|
||||
fun setup(
|
||||
applicationContext: WebApplicationContext,
|
||||
) {
|
||||
this.webClient = MockMvcWebTestClient
|
||||
.bindToApplicationContext(applicationContext).build()
|
||||
}
|
||||
|
||||
@Test
|
||||
fun testShouldRequestSSE() {
|
||||
statisticsUpdateProducer.emitComplete { _, _ -> true }
|
||||
|
||||
val result = webClient.get().uri("http://localhost/statistics/events").accept(TEXT_EVENT_STREAM).exchange()
|
||||
.expectStatus().isOk()
|
||||
.expectHeader().contentType(TEXT_EVENT_STREAM)
|
||||
.returnResult(String::class.java)
|
||||
|
||||
StepVerifier.create(result.responseBody)
|
||||
.expectComplete()
|
||||
.verify()
|
||||
}
|
||||
}
|
||||
|
||||
}
|
@ -23,41 +23,17 @@ import ca.uhn.fhir.context.FhirContext;
|
||||
import ca.uhn.fhir.parser.IParser;
|
||||
import dev.dnpm.etl.processor.config.GPasConfigProperties;
|
||||
import org.apache.commons.lang3.StringUtils;
|
||||
import org.apache.hc.client5.http.impl.classic.CloseableHttpClient;
|
||||
import org.apache.hc.client5.http.impl.classic.HttpClients;
|
||||
import org.apache.hc.client5.http.impl.io.BasicHttpClientConnectionManager;
|
||||
import org.apache.hc.client5.http.socket.ConnectionSocketFactory;
|
||||
import org.apache.hc.client5.http.socket.PlainConnectionSocketFactory;
|
||||
import org.apache.hc.client5.http.ssl.SSLConnectionSocketFactory;
|
||||
import org.apache.hc.core5.http.config.Registry;
|
||||
import org.apache.hc.core5.http.config.RegistryBuilder;
|
||||
import org.hl7.fhir.r4.model.Identifier;
|
||||
import org.hl7.fhir.r4.model.Parameters;
|
||||
import org.hl7.fhir.r4.model.Parameters.ParametersParameterComponent;
|
||||
import org.hl7.fhir.r4.model.StringType;
|
||||
import org.jetbrains.annotations.NotNull;
|
||||
import org.jetbrains.annotations.Nullable;
|
||||
import org.slf4j.Logger;
|
||||
import org.slf4j.LoggerFactory;
|
||||
import org.springframework.http.*;
|
||||
import org.springframework.http.client.HttpComponentsClientHttpRequestFactory;
|
||||
import org.springframework.retry.support.RetryTemplate;
|
||||
import org.springframework.web.client.RestTemplate;
|
||||
|
||||
import javax.net.ssl.SSLContext;
|
||||
import javax.net.ssl.TrustManagerFactory;
|
||||
import java.io.BufferedInputStream;
|
||||
import java.io.FileInputStream;
|
||||
import java.io.IOException;
|
||||
import java.security.KeyManagementException;
|
||||
import java.security.KeyStore;
|
||||
import java.security.KeyStoreException;
|
||||
import java.security.NoSuchAlgorithmException;
|
||||
import java.security.cert.CertificateException;
|
||||
import java.security.cert.CertificateFactory;
|
||||
import java.security.cert.X509Certificate;
|
||||
import java.util.Base64;
|
||||
|
||||
public class GpasPseudonymGenerator implements Generator {
|
||||
|
||||
private final static FhirContext r4Context = FhirContext.forR4();
|
||||
@ -69,27 +45,13 @@ public class GpasPseudonymGenerator implements Generator {
|
||||
|
||||
private final RestTemplate restTemplate;
|
||||
|
||||
private SSLContext customSslContext;
|
||||
|
||||
public GpasPseudonymGenerator(GPasConfigProperties gpasCfg, RetryTemplate retryTemplate) {
|
||||
public GpasPseudonymGenerator(GPasConfigProperties gpasCfg, RetryTemplate retryTemplate, RestTemplate restTemplate) {
|
||||
this.retryTemplate = retryTemplate;
|
||||
this.restTemplate = getRestTemplete();
|
||||
|
||||
this.restTemplate = restTemplate;
|
||||
this.gPasUrl = gpasCfg.getUri();
|
||||
this.psnTargetDomain = gpasCfg.getTarget();
|
||||
httpHeader = getHttpHeaders(gpasCfg.getUsername(), gpasCfg.getPassword());
|
||||
|
||||
try {
|
||||
if (StringUtils.isNotBlank(gpasCfg.getSslCaLocation())) {
|
||||
customSslContext = getSslContext(gpasCfg.getSslCaLocation());
|
||||
log.warn(String.format("%s has been initialized with SSL certificate %s. This is deprecated in favor of including Root CA.",
|
||||
this.getClass().getName(), gpasCfg.getSslCaLocation()));
|
||||
}
|
||||
} catch (IOException | KeyManagementException | KeyStoreException | CertificateException |
|
||||
NoSuchAlgorithmException e) {
|
||||
throw new RuntimeException(e);
|
||||
}
|
||||
|
||||
log.debug(String.format("%s has been initialized", this.getClass().getName()));
|
||||
|
||||
}
|
||||
@ -180,67 +142,7 @@ public class GpasPseudonymGenerator implements Generator {
|
||||
return headers;
|
||||
}
|
||||
|
||||
String authHeader = gPasUserName + ":" + gPasPassword;
|
||||
byte[] authHeaderBytes = authHeader.getBytes();
|
||||
byte[] encodedAuthHeaderBytes = Base64.getEncoder().encode(authHeaderBytes);
|
||||
String encodedAuthHeader = new String(encodedAuthHeaderBytes);
|
||||
|
||||
if (StringUtils.isNotBlank(gPasUserName) && StringUtils.isNotBlank(gPasPassword)) {
|
||||
headers.set("Authorization", "Basic " + encodedAuthHeader);
|
||||
}
|
||||
|
||||
headers.setBasicAuth(gPasUserName, gPasPassword);
|
||||
return headers;
|
||||
}
|
||||
|
||||
/**
|
||||
* Read SSL root certificate and return SSLContext
|
||||
*
|
||||
* @param certificateLocation file location to root certificate (PEM)
|
||||
* @return initialized SSLContext
|
||||
* @throws IOException file cannot be read
|
||||
* @throws CertificateException in case we have an invalid certificate of type X.509
|
||||
* @throws KeyStoreException keystore cannot be initialized
|
||||
* @throws NoSuchAlgorithmException missing trust manager algorithmus
|
||||
* @throws KeyManagementException key management failed at init SSLContext
|
||||
*/
|
||||
@Nullable
|
||||
protected SSLContext getSslContext(String certificateLocation)
|
||||
throws IOException, CertificateException, KeyStoreException, KeyManagementException, NoSuchAlgorithmException {
|
||||
|
||||
KeyStore ks = KeyStore.getInstance(KeyStore.getDefaultType());
|
||||
|
||||
FileInputStream fis = new FileInputStream(certificateLocation);
|
||||
X509Certificate ca = (X509Certificate) CertificateFactory.getInstance("X.509")
|
||||
.generateCertificate(new BufferedInputStream(fis));
|
||||
|
||||
ks.load(null, null);
|
||||
ks.setCertificateEntry(Integer.toString(1), ca);
|
||||
|
||||
TrustManagerFactory tmf = TrustManagerFactory.getInstance(
|
||||
TrustManagerFactory.getDefaultAlgorithm());
|
||||
tmf.init(ks);
|
||||
|
||||
SSLContext sslContext = SSLContext.getInstance("TLS");
|
||||
sslContext.init(null, tmf.getTrustManagers(), null);
|
||||
|
||||
return sslContext;
|
||||
}
|
||||
|
||||
protected RestTemplate getRestTemplete() {
|
||||
if (customSslContext == null) {
|
||||
return new RestTemplate();
|
||||
}
|
||||
final var sslsf = new SSLConnectionSocketFactory(customSslContext);
|
||||
final Registry<ConnectionSocketFactory> socketFactoryRegistry = RegistryBuilder.<ConnectionSocketFactory>create()
|
||||
.register("https", sslsf).register("http", new PlainConnectionSocketFactory()).build();
|
||||
|
||||
final BasicHttpClientConnectionManager connectionManager = new BasicHttpClientConnectionManager(
|
||||
socketFactoryRegistry);
|
||||
final CloseableHttpClient httpClient = HttpClients.custom()
|
||||
.setConnectionManager(connectionManager).build();
|
||||
|
||||
final HttpComponentsClientHttpRequestFactory requestFactory = new HttpComponentsClientHttpRequestFactory(
|
||||
httpClient);
|
||||
return new RestTemplate(requestFactory);
|
||||
}
|
||||
}
|
||||
|
@ -1,7 +1,7 @@
|
||||
/*
|
||||
* This file is part of ETL-Processor
|
||||
*
|
||||
* Copyright (c) 2024 Comprehensive Cancer Center Mainfranken, Datenintegrationszentrum Philipps-Universität Marburg and Contributors
|
||||
* Copyright (c) 2025 Comprehensive Cancer Center Mainfranken, Datenintegrationszentrum Philipps-Universität Marburg and Contributors
|
||||
*
|
||||
* This program is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
@ -21,16 +21,10 @@ package dev.dnpm.etl.processor.config
|
||||
|
||||
import dev.dnpm.etl.processor.security.Role
|
||||
import org.springframework.boot.context.properties.ConfigurationProperties
|
||||
import org.springframework.boot.context.properties.DeprecatedConfigurationProperty
|
||||
|
||||
@ConfigurationProperties(AppConfigProperties.NAME)
|
||||
data class AppConfigProperties(
|
||||
var bwhcUri: String?,
|
||||
@get:DeprecatedConfigurationProperty(
|
||||
reason = "Deprecated in favor of 'app.pseudonymize.generator'",
|
||||
replacement = "app.pseudonymize.generator"
|
||||
)
|
||||
var pseudonymizer: PseudonymGenerator = PseudonymGenerator.BUILDIN,
|
||||
var transformations: List<TransformationProperties> = listOf(),
|
||||
var maxRetryAttempts: Int = 3,
|
||||
var duplicationDetection: Boolean = true
|
||||
@ -56,10 +50,6 @@ data class GPasConfigProperties(
|
||||
val target: String = "etl-processor",
|
||||
val username: String?,
|
||||
val password: String?,
|
||||
@get:DeprecatedConfigurationProperty(
|
||||
reason = "Deprecated in favor of including Root CA"
|
||||
)
|
||||
val sslCaLocation: String?
|
||||
) {
|
||||
companion object {
|
||||
const val NAME = "app.pseudonymize.gpas"
|
||||
@ -69,6 +59,9 @@ data class GPasConfigProperties(
|
||||
@ConfigurationProperties(RestTargetProperties.NAME)
|
||||
data class RestTargetProperties(
|
||||
val uri: String?,
|
||||
val username: String?,
|
||||
val password: String?,
|
||||
val isBwhc: Boolean = false,
|
||||
) {
|
||||
companion object {
|
||||
const val NAME = "app.rest"
|
||||
@ -79,18 +72,8 @@ data class RestTargetProperties(
|
||||
data class KafkaProperties(
|
||||
val inputTopic: String?,
|
||||
val outputTopic: String = "etl-processor",
|
||||
@get:DeprecatedConfigurationProperty(
|
||||
reason = "Deprecated",
|
||||
replacement = "outputTopic"
|
||||
)
|
||||
val topic: String = outputTopic,
|
||||
val outputResponseTopic: String = "${outputTopic}_response",
|
||||
@get:DeprecatedConfigurationProperty(
|
||||
reason = "Deprecated",
|
||||
replacement = "outputResponseTopic"
|
||||
)
|
||||
val responseTopic: String = outputResponseTopic,
|
||||
val groupId: String = "${topic}_group",
|
||||
val groupId: String = "${outputTopic}_group",
|
||||
val servers: String = ""
|
||||
) {
|
||||
companion object {
|
||||
|
@ -1,7 +1,7 @@
|
||||
/*
|
||||
* This file is part of ETL-Processor
|
||||
*
|
||||
* Copyright (c) 2024 Comprehensive Cancer Center Mainfranken, Datenintegrationszentrum Philipps-Universität Marburg and Contributors
|
||||
* Copyright (c) 2025 Comprehensive Cancer Center Mainfranken, Datenintegrationszentrum Philipps-Universität Marburg and Contributors
|
||||
*
|
||||
* This program is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
@ -20,13 +20,16 @@
|
||||
package dev.dnpm.etl.processor.config
|
||||
|
||||
import com.fasterxml.jackson.databind.ObjectMapper
|
||||
import dev.dnpm.etl.processor.monitoring.*
|
||||
import dev.dnpm.etl.processor.monitoring.ConnectionCheckResult
|
||||
import dev.dnpm.etl.processor.monitoring.ConnectionCheckService
|
||||
import dev.dnpm.etl.processor.monitoring.GPasConnectionCheckService
|
||||
import dev.dnpm.etl.processor.monitoring.ReportService
|
||||
import dev.dnpm.etl.processor.pseudonym.AnonymizingGenerator
|
||||
import dev.dnpm.etl.processor.pseudonym.Generator
|
||||
import dev.dnpm.etl.processor.pseudonym.GpasPseudonymGenerator
|
||||
import dev.dnpm.etl.processor.pseudonym.PseudonymizeService
|
||||
import dev.dnpm.etl.processor.services.TokenRepository
|
||||
import dev.dnpm.etl.processor.services.TokenService
|
||||
import dev.dnpm.etl.processor.security.TokenRepository
|
||||
import dev.dnpm.etl.processor.security.TokenService
|
||||
import dev.dnpm.etl.processor.services.Transformation
|
||||
import dev.dnpm.etl.processor.services.TransformationService
|
||||
import org.slf4j.LoggerFactory
|
||||
@ -35,6 +38,7 @@ import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty
|
||||
import org.springframework.boot.context.properties.EnableConfigurationProperties
|
||||
import org.springframework.context.annotation.Bean
|
||||
import org.springframework.context.annotation.Configuration
|
||||
import org.springframework.data.jdbc.repository.config.AbstractJdbcConfiguration
|
||||
import org.springframework.retry.RetryCallback
|
||||
import org.springframework.retry.RetryContext
|
||||
import org.springframework.retry.RetryListener
|
||||
@ -44,6 +48,7 @@ import org.springframework.retry.support.RetryTemplateBuilder
|
||||
import org.springframework.scheduling.annotation.EnableScheduling
|
||||
import org.springframework.security.crypto.password.PasswordEncoder
|
||||
import org.springframework.security.provisioning.InMemoryUserDetailsManager
|
||||
import org.springframework.web.client.HttpClientErrorException
|
||||
import org.springframework.web.client.RestTemplate
|
||||
import reactor.core.publisher.Sinks
|
||||
import kotlin.time.Duration.Companion.seconds
|
||||
@ -70,8 +75,8 @@ class AppConfiguration {
|
||||
|
||||
@ConditionalOnProperty(value = ["app.pseudonymize.generator"], havingValue = "GPAS")
|
||||
@Bean
|
||||
fun gpasPseudonymGenerator(configProperties: GPasConfigProperties, retryTemplate: RetryTemplate): Generator {
|
||||
return GpasPseudonymGenerator(configProperties, retryTemplate)
|
||||
fun gpasPseudonymGenerator(configProperties: GPasConfigProperties, retryTemplate: RetryTemplate, restTemplate: RestTemplate): Generator {
|
||||
return GpasPseudonymGenerator(configProperties, retryTemplate, restTemplate)
|
||||
}
|
||||
|
||||
@ConditionalOnProperty(value = ["app.pseudonymize.generator"], havingValue = "BUILDIN", matchIfMissing = true)
|
||||
@ -80,20 +85,6 @@ class AppConfiguration {
|
||||
return AnonymizingGenerator()
|
||||
}
|
||||
|
||||
@ConditionalOnProperty(value = ["app.pseudonymizer"], havingValue = "GPAS")
|
||||
@ConditionalOnMissingBean
|
||||
@Bean
|
||||
fun gpasPseudonymGeneratorOnDeprecatedProperty(configProperties: GPasConfigProperties, retryTemplate: RetryTemplate): Generator {
|
||||
return GpasPseudonymGenerator(configProperties, retryTemplate)
|
||||
}
|
||||
|
||||
@ConditionalOnProperty(value = ["app.pseudonymizer"], havingValue = "BUILDIN")
|
||||
@ConditionalOnMissingBean
|
||||
@Bean
|
||||
fun buildinPseudonymGeneratorOnDeprecatedProperty(): Generator {
|
||||
return AnonymizingGenerator()
|
||||
}
|
||||
|
||||
@Bean
|
||||
fun pseudonymizeService(
|
||||
generator: Generator,
|
||||
@ -122,6 +113,8 @@ class AppConfiguration {
|
||||
fun retryTemplate(configProperties: AppConfigProperties): RetryTemplate {
|
||||
return RetryTemplateBuilder()
|
||||
.notRetryOn(IllegalArgumentException::class.java)
|
||||
.notRetryOn(HttpClientErrorException.BadRequest::class.java)
|
||||
.notRetryOn(HttpClientErrorException.UnprocessableEntity::class.java)
|
||||
.exponentialBackoff(2.seconds.toJavaDuration(), 1.25, 5.seconds.toJavaDuration())
|
||||
.customPolicy(SimpleRetryPolicy(configProperties.maxRetryAttempts))
|
||||
.withListener(object : RetryListener {
|
||||
@ -173,5 +166,9 @@ class AppConfiguration {
|
||||
return GPasConnectionCheckService(restTemplate, gPasConfigProperties, connectionCheckUpdateProducer)
|
||||
}
|
||||
|
||||
@Bean
|
||||
fun jdbcConfiguration(): AbstractJdbcConfiguration {
|
||||
return AppJdbcConfiguration()
|
||||
}
|
||||
}
|
||||
|
||||
|
@ -0,0 +1,25 @@
|
||||
package dev.dnpm.etl.processor.config
|
||||
|
||||
import dev.dnpm.etl.processor.Fingerprint
|
||||
import org.springframework.context.annotation.Configuration
|
||||
import org.springframework.core.convert.converter.Converter
|
||||
import org.springframework.data.jdbc.repository.config.AbstractJdbcConfiguration
|
||||
|
||||
@Configuration
|
||||
class AppJdbcConfiguration : AbstractJdbcConfiguration() {
|
||||
override fun userConverters(): MutableList<*> {
|
||||
return mutableListOf(StringToFingerprintConverter(), FingerprintToStringConverter())
|
||||
}
|
||||
}
|
||||
|
||||
class StringToFingerprintConverter : Converter<String, Fingerprint> {
|
||||
override fun convert(source: String): Fingerprint {
|
||||
return Fingerprint(source)
|
||||
}
|
||||
}
|
||||
|
||||
class FingerprintToStringConverter : Converter<Fingerprint, String> {
|
||||
override fun convert(source: Fingerprint): String {
|
||||
return source.value
|
||||
}
|
||||
}
|
@ -1,7 +1,7 @@
|
||||
/*
|
||||
* This file is part of ETL-Processor
|
||||
*
|
||||
* Copyright (c) 2024 Comprehensive Cancer Center Mainfranken, Datenintegrationszentrum Philipps-Universität Marburg and Contributors
|
||||
* Copyright (c) 2025 Comprehensive Cancer Center Mainfranken, Datenintegrationszentrum Philipps-Universität Marburg and Contributors
|
||||
*
|
||||
* This program is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
@ -71,7 +71,7 @@ class AppKafkaConfiguration {
|
||||
kafkaProperties: KafkaProperties,
|
||||
kafkaResponseProcessor: KafkaResponseProcessor
|
||||
): KafkaMessageListenerContainer<String, String> {
|
||||
val containerProperties = ContainerProperties(kafkaProperties.responseTopic)
|
||||
val containerProperties = ContainerProperties(kafkaProperties.outputResponseTopic)
|
||||
containerProperties.messageListener = kafkaResponseProcessor
|
||||
return KafkaMessageListenerContainer(consumerFactory, containerProperties)
|
||||
}
|
||||
|
@ -1,7 +1,7 @@
|
||||
/*
|
||||
* This file is part of ETL-Processor
|
||||
*
|
||||
* Copyright (c) 2024 Comprehensive Cancer Center Mainfranken, Datenintegrationszentrum Philipps-Universität Marburg and Contributors
|
||||
* Copyright (c) 2025 Comprehensive Cancer Center Mainfranken, Datenintegrationszentrum Philipps-Universität Marburg and Contributors
|
||||
*
|
||||
* This program is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
@ -21,9 +21,11 @@ package dev.dnpm.etl.processor.config
|
||||
|
||||
import dev.dnpm.etl.processor.monitoring.ConnectionCheckResult
|
||||
import dev.dnpm.etl.processor.monitoring.ConnectionCheckService
|
||||
import dev.dnpm.etl.processor.monitoring.ReportService
|
||||
import dev.dnpm.etl.processor.monitoring.RestConnectionCheckService
|
||||
import dev.dnpm.etl.processor.output.MtbFileSender
|
||||
import dev.dnpm.etl.processor.output.RestMtbFileSender
|
||||
import dev.dnpm.etl.processor.output.RestBwhcMtbFileSender
|
||||
import dev.dnpm.etl.processor.output.RestDipMtbFileSender
|
||||
import org.slf4j.LoggerFactory
|
||||
import org.springframework.boot.autoconfigure.condition.ConditionalOnMissingBean
|
||||
import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty
|
||||
@ -52,10 +54,16 @@ class AppRestConfiguration {
|
||||
fun restMtbFileSender(
|
||||
restTemplate: RestTemplate,
|
||||
restTargetProperties: RestTargetProperties,
|
||||
retryTemplate: RetryTemplate
|
||||
retryTemplate: RetryTemplate,
|
||||
reportService: ReportService,
|
||||
): MtbFileSender {
|
||||
logger.info("Selected 'RestMtbFileSender'")
|
||||
return RestMtbFileSender(restTemplate, restTargetProperties, retryTemplate)
|
||||
if (restTargetProperties.isBwhc) {
|
||||
logger.info("Selected 'RestBwhcMtbFileSender'")
|
||||
return RestBwhcMtbFileSender(restTemplate, restTargetProperties, retryTemplate, reportService)
|
||||
}
|
||||
|
||||
logger.info("Selected 'RestDipMtbFileSender'")
|
||||
return RestDipMtbFileSender(restTemplate, restTargetProperties, retryTemplate, reportService)
|
||||
}
|
||||
|
||||
@Bean
|
||||
|
@ -1,7 +1,7 @@
|
||||
/*
|
||||
* This file is part of ETL-Processor
|
||||
*
|
||||
* Copyright (c) 2023 Comprehensive Cancer Center Mainfranken, Datenintegrationszentrum Philipps-Universität Marburg and Contributors
|
||||
* Copyright (c) 2025 Comprehensive Cancer Center Mainfranken, Datenintegrationszentrum Philipps-Universität Marburg and Contributors
|
||||
*
|
||||
* This program is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
@ -21,7 +21,7 @@ package dev.dnpm.etl.processor.config
|
||||
|
||||
import dev.dnpm.etl.processor.security.UserRole
|
||||
import dev.dnpm.etl.processor.security.UserRoleRepository
|
||||
import dev.dnpm.etl.processor.services.UserRoleService
|
||||
import dev.dnpm.etl.processor.security.UserRoleService
|
||||
import org.slf4j.LoggerFactory
|
||||
import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty
|
||||
import org.springframework.boot.context.properties.EnableConfigurationProperties
|
||||
@ -44,6 +44,8 @@ import org.springframework.security.web.SecurityFilterChain
|
||||
import java.util.*
|
||||
|
||||
|
||||
private const val LOGIN_PATH = "/login"
|
||||
|
||||
@Configuration
|
||||
@EnableConfigurationProperties(
|
||||
value = [
|
||||
@ -85,11 +87,16 @@ class AppSecurityConfiguration(
|
||||
|
||||
@Bean
|
||||
@ConditionalOnProperty(value = ["app.security.enable-oidc"], havingValue = "true")
|
||||
fun filterChainOidc(http: HttpSecurity, passwordEncoder: PasswordEncoder, userRoleRepository: UserRoleRepository, sessionRegistry: SessionRegistry): SecurityFilterChain {
|
||||
fun filterChainOidc(
|
||||
http: HttpSecurity,
|
||||
passwordEncoder: PasswordEncoder,
|
||||
userRoleRepository: UserRoleRepository,
|
||||
sessionRegistry: SessionRegistry
|
||||
): SecurityFilterChain {
|
||||
http {
|
||||
authorizeRequests {
|
||||
authorizeHttpRequests {
|
||||
authorize("/configs/**", hasRole("ADMIN"))
|
||||
authorize("/mtbfile/**", hasAnyRole("MTBFILE"))
|
||||
authorize("/mtbfile/**", hasAnyRole("MTBFILE", "ADMIN", "USER"))
|
||||
authorize("/report/**", hasAnyRole("ADMIN", "USER"))
|
||||
authorize("*.css", permitAll)
|
||||
authorize("*.ico", permitAll)
|
||||
@ -104,15 +111,15 @@ class AppSecurityConfiguration(
|
||||
realmName = "ETL-Processor"
|
||||
}
|
||||
formLogin {
|
||||
loginPage = "/login"
|
||||
loginPage = LOGIN_PATH
|
||||
}
|
||||
oauth2Login {
|
||||
loginPage = "/login"
|
||||
loginPage = LOGIN_PATH
|
||||
}
|
||||
sessionManagement {
|
||||
sessionConcurrency {
|
||||
maximumSessions = 1
|
||||
expiredUrl = "/login?expired"
|
||||
expiredUrl = "$LOGIN_PATH?expired"
|
||||
}
|
||||
sessionFixation {
|
||||
newSession()
|
||||
@ -125,13 +132,22 @@ class AppSecurityConfiguration(
|
||||
|
||||
@Bean
|
||||
@ConditionalOnProperty(value = ["app.security.enable-oidc"], havingValue = "true")
|
||||
fun grantedAuthoritiesMapper(userRoleRepository: UserRoleRepository, appSecurityConfigProperties: SecurityConfigProperties): GrantedAuthoritiesMapper {
|
||||
fun grantedAuthoritiesMapper(
|
||||
userRoleRepository: UserRoleRepository,
|
||||
appSecurityConfigProperties: SecurityConfigProperties
|
||||
): GrantedAuthoritiesMapper {
|
||||
return GrantedAuthoritiesMapper { grantedAuthority ->
|
||||
grantedAuthority.filterIsInstance<OidcUserAuthority>()
|
||||
.onEach {
|
||||
val userRole = userRoleRepository.findByUsername(it.userInfo.preferredUsername)
|
||||
if (userRole.isEmpty) {
|
||||
userRoleRepository.save(UserRole(null, it.userInfo.preferredUsername, appSecurityConfigProperties.defaultNewUserRole))
|
||||
userRoleRepository.save(
|
||||
UserRole(
|
||||
null,
|
||||
it.userInfo.preferredUsername,
|
||||
appSecurityConfigProperties.defaultNewUserRole
|
||||
)
|
||||
)
|
||||
}
|
||||
}
|
||||
.map {
|
||||
@ -145,9 +161,9 @@ class AppSecurityConfiguration(
|
||||
@ConditionalOnProperty(value = ["app.security.enable-oidc"], havingValue = "false", matchIfMissing = true)
|
||||
fun filterChain(http: HttpSecurity, passwordEncoder: PasswordEncoder): SecurityFilterChain {
|
||||
http {
|
||||
authorizeRequests {
|
||||
authorizeHttpRequests {
|
||||
authorize("/configs/**", hasRole("ADMIN"))
|
||||
authorize("/mtbfile/**", hasAnyRole("MTBFILE"))
|
||||
authorize("/mtbfile/**", hasAnyRole("MTBFILE", "ADMIN"))
|
||||
authorize("/report/**", hasRole("ADMIN"))
|
||||
authorize(anyRequest, permitAll)
|
||||
}
|
||||
@ -155,7 +171,7 @@ class AppSecurityConfiguration(
|
||||
realmName = "ETL-Processor"
|
||||
}
|
||||
formLogin {
|
||||
loginPage = "/login"
|
||||
loginPage = LOGIN_PATH
|
||||
}
|
||||
csrf { disable() }
|
||||
}
|
||||
|
@ -1,7 +1,7 @@
|
||||
/*
|
||||
* This file is part of ETL-Processor
|
||||
*
|
||||
* Copyright (c) 2024 Comprehensive Cancer Center Mainfranken, Datenintegrationszentrum Philipps-Universität Marburg and Contributors
|
||||
* Copyright (c) 2025 Comprehensive Cancer Center Mainfranken, Datenintegrationszentrum Philipps-Universität Marburg and Contributors
|
||||
*
|
||||
* This program is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
@ -22,9 +22,13 @@ package dev.dnpm.etl.processor.input
|
||||
import com.fasterxml.jackson.databind.ObjectMapper
|
||||
import de.ukw.ccc.bwhc.dto.Consent
|
||||
import de.ukw.ccc.bwhc.dto.MtbFile
|
||||
import dev.dnpm.etl.processor.CustomMediaType
|
||||
import dev.dnpm.etl.processor.PatientId
|
||||
import dev.dnpm.etl.processor.RequestId
|
||||
import dev.dnpm.etl.processor.services.RequestProcessor
|
||||
import org.apache.kafka.clients.consumer.ConsumerRecord
|
||||
import org.slf4j.LoggerFactory
|
||||
import org.springframework.http.MediaType
|
||||
import org.springframework.kafka.listener.MessageListener
|
||||
|
||||
class KafkaInputListener(
|
||||
@ -33,13 +37,33 @@ class KafkaInputListener(
|
||||
) : MessageListener<String, String> {
|
||||
private val logger = LoggerFactory.getLogger(KafkaInputListener::class.java)
|
||||
|
||||
override fun onMessage(data: ConsumerRecord<String, String>) {
|
||||
val mtbFile = objectMapper.readValue(data.value(), MtbFile::class.java)
|
||||
val firstRequestIdHeader = data.headers().headers("requestId")?.firstOrNull()
|
||||
override fun onMessage(record: ConsumerRecord<String, String>) {
|
||||
when (guessMimeType(record)) {
|
||||
MediaType.APPLICATION_JSON_VALUE -> handleBwhcMessage(record)
|
||||
CustomMediaType.APPLICATION_VND_DNPM_V2_MTB_JSON_VALUE -> handleDnpmV2Message(record)
|
||||
else -> {
|
||||
/* ignore other messages */
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private fun guessMimeType(record: ConsumerRecord<String, String>): String {
|
||||
if (record.headers().headers("contentType").toList().isEmpty()) {
|
||||
// Fallback if no contentType set (old behavior)
|
||||
return MediaType.APPLICATION_JSON_VALUE
|
||||
}
|
||||
|
||||
return record.headers().headers("contentType")?.firstOrNull()?.value().contentToString()
|
||||
}
|
||||
|
||||
private fun handleBwhcMessage(record: ConsumerRecord<String, String>) {
|
||||
val mtbFile = objectMapper.readValue(record.value(), MtbFile::class.java)
|
||||
val patientId = PatientId(mtbFile.patient.id)
|
||||
val firstRequestIdHeader = record.headers().headers("requestId")?.firstOrNull()
|
||||
val requestId = if (null != firstRequestIdHeader) {
|
||||
String(firstRequestIdHeader.value())
|
||||
RequestId(String(firstRequestIdHeader.value()))
|
||||
} else {
|
||||
""
|
||||
RequestId("")
|
||||
}
|
||||
|
||||
if (mtbFile.consent.status == Consent.Status.ACTIVE) {
|
||||
@ -52,10 +76,16 @@ class KafkaInputListener(
|
||||
} else {
|
||||
logger.debug("Accepted MTB File and process deletion")
|
||||
if (requestId.isBlank()) {
|
||||
requestProcessor.processDeletion(mtbFile.patient.id)
|
||||
requestProcessor.processDeletion(patientId)
|
||||
} else {
|
||||
requestProcessor.processDeletion(mtbFile.patient.id, requestId)
|
||||
requestProcessor.processDeletion(patientId, requestId)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private fun handleDnpmV2Message(record: ConsumerRecord<String, String>) {
|
||||
// Do not handle DNPM-V2 for now
|
||||
logger.warn("Ignoring MTB File in DNPM V2 format: Not implemented yet")
|
||||
}
|
||||
|
||||
}
|
@ -1,7 +1,7 @@
|
||||
/*
|
||||
* This file is part of ETL-Processor
|
||||
*
|
||||
* Copyright (c) 2024 Comprehensive Cancer Center Mainfranken, Datenintegrationszentrum Philipps-Universität Marburg and Contributors
|
||||
* Copyright (c) 2025 Comprehensive Cancer Center Mainfranken, Datenintegrationszentrum Philipps-Universität Marburg and Contributors
|
||||
*
|
||||
* This program is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
@ -21,13 +21,17 @@ package dev.dnpm.etl.processor.input
|
||||
|
||||
import de.ukw.ccc.bwhc.dto.Consent
|
||||
import de.ukw.ccc.bwhc.dto.MtbFile
|
||||
import dev.dnpm.etl.processor.CustomMediaType
|
||||
import dev.dnpm.etl.processor.PatientId
|
||||
import dev.dnpm.etl.processor.services.RequestProcessor
|
||||
import dev.pcvolkmer.mv64e.mtb.Mtb
|
||||
import org.slf4j.LoggerFactory
|
||||
import org.springframework.http.MediaType
|
||||
import org.springframework.http.ResponseEntity
|
||||
import org.springframework.web.bind.annotation.*
|
||||
|
||||
@RestController
|
||||
@RequestMapping(path = ["mtbfile"])
|
||||
@RequestMapping(path = ["mtbfile", "mtb"])
|
||||
class MtbFileRestController(
|
||||
private val requestProcessor: RequestProcessor,
|
||||
) {
|
||||
@ -39,22 +43,30 @@ class MtbFileRestController(
|
||||
return ResponseEntity.ok("Test")
|
||||
}
|
||||
|
||||
@PostMapping
|
||||
fun mtbFile(@RequestBody mtbFile: MtbFile): ResponseEntity<Void> {
|
||||
@PostMapping( consumes = [ MediaType.APPLICATION_JSON_VALUE ] )
|
||||
fun mtbFile(@RequestBody mtbFile: MtbFile): ResponseEntity<Unit> {
|
||||
if (mtbFile.consent.status == Consent.Status.ACTIVE) {
|
||||
logger.debug("Accepted MTB File for processing")
|
||||
logger.debug("Accepted MTB File (bwHC V1) for processing")
|
||||
requestProcessor.processMtbFile(mtbFile)
|
||||
} else {
|
||||
logger.debug("Accepted MTB File and process deletion")
|
||||
requestProcessor.processDeletion(mtbFile.patient.id)
|
||||
logger.debug("Accepted MTB File (bwHC V1) and process deletion")
|
||||
val patientId = PatientId(mtbFile.patient.id)
|
||||
requestProcessor.processDeletion(patientId)
|
||||
}
|
||||
return ResponseEntity.accepted().build()
|
||||
}
|
||||
|
||||
@PostMapping( consumes = [ CustomMediaType.APPLICATION_VND_DNPM_V2_MTB_JSON_VALUE] )
|
||||
fun mtbFile(@RequestBody mtbFile: Mtb): ResponseEntity<Unit> {
|
||||
logger.debug("Accepted MTB File (DNPM V2) for processing")
|
||||
requestProcessor.processMtbFile(mtbFile)
|
||||
return ResponseEntity.accepted().build()
|
||||
}
|
||||
|
||||
@DeleteMapping(path = ["{patientId}"])
|
||||
fun deleteData(@PathVariable patientId: String): ResponseEntity<Void> {
|
||||
fun deleteData(@PathVariable patientId: String): ResponseEntity<Unit> {
|
||||
logger.debug("Accepted patient ID to process deletion")
|
||||
requestProcessor.processDeletion(patientId)
|
||||
requestProcessor.processDeletion(PatientId(patientId))
|
||||
return ResponseEntity.accepted().build()
|
||||
}
|
||||
|
||||
|
@ -26,22 +26,18 @@ import jakarta.annotation.PostConstruct
|
||||
import org.apache.kafka.clients.consumer.Consumer
|
||||
import org.apache.kafka.common.errors.TimeoutException
|
||||
import org.springframework.beans.factory.annotation.Qualifier
|
||||
import org.springframework.http.HttpEntity
|
||||
import org.springframework.http.HttpHeaders
|
||||
import org.springframework.http.HttpMethod
|
||||
import org.springframework.http.HttpStatus
|
||||
import org.springframework.http.MediaType
|
||||
import org.springframework.http.RequestEntity
|
||||
import org.springframework.http.*
|
||||
import org.springframework.scheduling.annotation.Scheduled
|
||||
import org.springframework.web.client.RestTemplate
|
||||
import org.springframework.web.util.UriComponentsBuilder
|
||||
import reactor.core.publisher.Sinks
|
||||
import java.time.Instant
|
||||
import kotlin.time.Duration.Companion.seconds
|
||||
import kotlin.time.toJavaDuration
|
||||
|
||||
interface ConnectionCheckService {
|
||||
fun interface ConnectionCheckService {
|
||||
|
||||
fun connectionAvailable(): Boolean
|
||||
fun connectionAvailable(): ConnectionCheckResult
|
||||
|
||||
}
|
||||
|
||||
@ -51,9 +47,27 @@ sealed class ConnectionCheckResult {
|
||||
|
||||
abstract val available: Boolean
|
||||
|
||||
data class KafkaConnectionCheckResult(override val available: Boolean) : ConnectionCheckResult()
|
||||
data class RestConnectionCheckResult(override val available: Boolean) : ConnectionCheckResult()
|
||||
data class GPasConnectionCheckResult(override val available: Boolean) : ConnectionCheckResult()
|
||||
abstract val timestamp: Instant
|
||||
|
||||
abstract val lastChange: Instant
|
||||
|
||||
data class KafkaConnectionCheckResult(
|
||||
override val available: Boolean,
|
||||
override val timestamp: Instant,
|
||||
override val lastChange: Instant
|
||||
) : ConnectionCheckResult()
|
||||
|
||||
data class RestConnectionCheckResult(
|
||||
override val available: Boolean,
|
||||
override val timestamp: Instant,
|
||||
override val lastChange: Instant
|
||||
) : ConnectionCheckResult()
|
||||
|
||||
data class GPasConnectionCheckResult(
|
||||
override val available: Boolean,
|
||||
override val timestamp: Instant,
|
||||
override val lastChange: Instant
|
||||
) : ConnectionCheckResult()
|
||||
}
|
||||
|
||||
class KafkaConnectionCheckService(
|
||||
@ -62,25 +76,33 @@ class KafkaConnectionCheckService(
|
||||
private val connectionCheckUpdateProducer: Sinks.Many<ConnectionCheckResult>
|
||||
) : OutputConnectionCheckService {
|
||||
|
||||
private var connectionAvailable: Boolean = false
|
||||
|
||||
private var result = ConnectionCheckResult.KafkaConnectionCheckResult(false, Instant.now(), Instant.now())
|
||||
|
||||
@PostConstruct
|
||||
@Scheduled(cron = "0 * * * * *")
|
||||
fun check() {
|
||||
connectionAvailable = try {
|
||||
null != consumer.listTopics(5.seconds.toJavaDuration())
|
||||
} catch (e: TimeoutException) {
|
||||
false
|
||||
result = try {
|
||||
val available = null != consumer.listTopics(5.seconds.toJavaDuration())
|
||||
ConnectionCheckResult.KafkaConnectionCheckResult(
|
||||
available,
|
||||
Instant.now(),
|
||||
if (result.available == available) { result.lastChange } else { Instant.now() }
|
||||
)
|
||||
} catch (_: TimeoutException) {
|
||||
ConnectionCheckResult.KafkaConnectionCheckResult(
|
||||
false,
|
||||
Instant.now(),
|
||||
if (!result.available) { result.lastChange } else { Instant.now() }
|
||||
)
|
||||
}
|
||||
connectionCheckUpdateProducer.emitNext(
|
||||
ConnectionCheckResult.KafkaConnectionCheckResult(connectionAvailable),
|
||||
result,
|
||||
Sinks.EmitFailureHandler.FAIL_FAST
|
||||
)
|
||||
}
|
||||
|
||||
override fun connectionAvailable(): Boolean {
|
||||
return this.connectionAvailable
|
||||
override fun connectionAvailable(): ConnectionCheckResult.KafkaConnectionCheckResult {
|
||||
return this.result
|
||||
}
|
||||
|
||||
}
|
||||
@ -92,27 +114,45 @@ class RestConnectionCheckService(
|
||||
private val connectionCheckUpdateProducer: Sinks.Many<ConnectionCheckResult>
|
||||
) : OutputConnectionCheckService {
|
||||
|
||||
private var connectionAvailable: Boolean = false
|
||||
private var result = ConnectionCheckResult.RestConnectionCheckResult(false, Instant.now(), Instant.now())
|
||||
|
||||
@PostConstruct
|
||||
@Scheduled(cron = "0 * * * * *")
|
||||
fun check() {
|
||||
connectionAvailable = try {
|
||||
restTemplate.getForEntity(
|
||||
restTargetProperties.uri?.replace("/etl/api", "").toString(),
|
||||
result = try {
|
||||
val available = restTemplate.getForEntity(
|
||||
if (restTargetProperties.isBwhc) {
|
||||
UriComponentsBuilder.fromUriString(restTargetProperties.uri.toString()).path("").toUriString()
|
||||
} else {
|
||||
UriComponentsBuilder.fromUriString(restTargetProperties.uri.toString())
|
||||
.pathSegment("mtb")
|
||||
.pathSegment("kaplan-meier")
|
||||
.pathSegment("config")
|
||||
.toUriString()
|
||||
},
|
||||
String::class.java
|
||||
).statusCode == HttpStatus.OK
|
||||
} catch (e: Exception) {
|
||||
false
|
||||
|
||||
ConnectionCheckResult.RestConnectionCheckResult(
|
||||
available,
|
||||
Instant.now(),
|
||||
if (result.available == available) { result.lastChange } else { Instant.now() }
|
||||
)
|
||||
} catch (_: Exception) {
|
||||
ConnectionCheckResult.RestConnectionCheckResult(
|
||||
false,
|
||||
Instant.now(),
|
||||
if (!result.available) { result.lastChange } else { Instant.now() }
|
||||
)
|
||||
}
|
||||
connectionCheckUpdateProducer.emitNext(
|
||||
ConnectionCheckResult.RestConnectionCheckResult(connectionAvailable),
|
||||
result,
|
||||
Sinks.EmitFailureHandler.FAIL_FAST
|
||||
)
|
||||
}
|
||||
|
||||
override fun connectionAvailable(): Boolean {
|
||||
return this.connectionAvailable
|
||||
override fun connectionAvailable(): ConnectionCheckResult.RestConnectionCheckResult {
|
||||
return this.result
|
||||
}
|
||||
}
|
||||
|
||||
@ -123,40 +163,48 @@ class GPasConnectionCheckService(
|
||||
private val connectionCheckUpdateProducer: Sinks.Many<ConnectionCheckResult>
|
||||
) : ConnectionCheckService {
|
||||
|
||||
private var connectionAvailable: Boolean = false
|
||||
private var result = ConnectionCheckResult.GPasConnectionCheckResult(false, Instant.now(), Instant.now())
|
||||
|
||||
@PostConstruct
|
||||
@Scheduled(cron = "0 * * * * *")
|
||||
fun check() {
|
||||
connectionAvailable = try {
|
||||
result = try {
|
||||
val uri = UriComponentsBuilder.fromUriString(
|
||||
gPasConfigProperties.uri?.replace("/\$pseudonymizeAllowCreate", "/\$pseudonymize").toString()
|
||||
)
|
||||
.queryParam("target", gPasConfigProperties.target)
|
||||
.queryParam("original", "???")
|
||||
.build().toUri()
|
||||
gPasConfigProperties.uri?.replace("/\$pseudonymizeAllowCreate", "/metadata").toString()
|
||||
).build().toUri()
|
||||
|
||||
val headers = HttpHeaders()
|
||||
headers.contentType = MediaType.APPLICATION_JSON
|
||||
if (!gPasConfigProperties.username.isNullOrBlank() && !gPasConfigProperties.password.isNullOrBlank()) {
|
||||
headers.setBasicAuth(gPasConfigProperties.username, gPasConfigProperties.password)
|
||||
}
|
||||
restTemplate.exchange(
|
||||
|
||||
val available = restTemplate.exchange(
|
||||
uri,
|
||||
HttpMethod.GET,
|
||||
HttpEntity<Void>(headers),
|
||||
Void::class.java
|
||||
).statusCode == HttpStatus.OK
|
||||
} catch (e: Exception) {
|
||||
false
|
||||
|
||||
ConnectionCheckResult.GPasConnectionCheckResult(
|
||||
available,
|
||||
Instant.now(),
|
||||
if (result.available == available) { result.lastChange } else { Instant.now() }
|
||||
)
|
||||
} catch (_: Exception) {
|
||||
ConnectionCheckResult.GPasConnectionCheckResult(
|
||||
false,
|
||||
Instant.now(),
|
||||
if (!result.available) { result.lastChange } else { Instant.now() }
|
||||
)
|
||||
}
|
||||
connectionCheckUpdateProducer.emitNext(
|
||||
ConnectionCheckResult.GPasConnectionCheckResult(connectionAvailable),
|
||||
result,
|
||||
Sinks.EmitFailureHandler.FAIL_FAST
|
||||
)
|
||||
}
|
||||
|
||||
override fun connectionAvailable(): Boolean {
|
||||
return this.connectionAvailable
|
||||
override fun connectionAvailable(): ConnectionCheckResult.GPasConnectionCheckResult {
|
||||
return this.result
|
||||
}
|
||||
}
|
@ -19,11 +19,15 @@
|
||||
|
||||
package dev.dnpm.etl.processor.monitoring
|
||||
|
||||
import com.fasterxml.jackson.annotation.JsonAlias
|
||||
import com.fasterxml.jackson.annotation.JsonIgnoreProperties
|
||||
import com.fasterxml.jackson.annotation.JsonValue
|
||||
import com.fasterxml.jackson.core.JsonParseException
|
||||
import com.fasterxml.jackson.databind.JsonMappingException
|
||||
import com.fasterxml.jackson.databind.ObjectMapper
|
||||
import dev.dnpm.etl.processor.monitoring.ReportService.Issue
|
||||
import dev.dnpm.etl.processor.monitoring.ReportService.Severity
|
||||
import java.util.Optional
|
||||
|
||||
class ReportService(
|
||||
private val objectMapper: ObjectMapper
|
||||
@ -54,7 +58,11 @@ class ReportService(
|
||||
private data class DataQualityReport(val issues: List<Issue>)
|
||||
|
||||
@JsonIgnoreProperties(ignoreUnknown = true)
|
||||
data class Issue(val severity: Severity, val message: String)
|
||||
data class Issue(
|
||||
val severity: Severity,
|
||||
@JsonAlias("details") val message: String,
|
||||
val path: Optional<String> = Optional.empty()
|
||||
)
|
||||
|
||||
enum class Severity(@JsonValue val value: String) {
|
||||
FATAL("fatal"),
|
||||
@ -63,3 +71,12 @@ class ReportService(
|
||||
INFO("info")
|
||||
}
|
||||
}
|
||||
|
||||
fun List<Issue>.asRequestStatus(): RequestStatus {
|
||||
val severity = this.minOfOrNull { it.severity }
|
||||
return when (severity) {
|
||||
Severity.FATAL, Severity.ERROR -> RequestStatus.ERROR
|
||||
Severity.WARNING -> RequestStatus.WARNING
|
||||
else -> RequestStatus.SUCCESS
|
||||
}
|
||||
}
|
@ -19,10 +19,12 @@
|
||||
|
||||
package dev.dnpm.etl.processor.monitoring
|
||||
|
||||
import dev.dnpm.etl.processor.*
|
||||
import org.springframework.data.annotation.Id
|
||||
import org.springframework.data.domain.Page
|
||||
import org.springframework.data.domain.Pageable
|
||||
import org.springframework.data.jdbc.repository.query.Query
|
||||
import org.springframework.data.relational.core.mapping.Column
|
||||
import org.springframework.data.relational.core.mapping.Embedded
|
||||
import org.springframework.data.relational.core.mapping.Table
|
||||
import org.springframework.data.repository.CrudRepository
|
||||
@ -30,26 +32,48 @@ import org.springframework.data.repository.PagingAndSortingRepository
|
||||
import java.time.Instant
|
||||
import java.util.*
|
||||
|
||||
typealias RequestId = UUID
|
||||
|
||||
@Table("request")
|
||||
data class Request(
|
||||
@Id val id: Long? = null,
|
||||
val uuid: String = RequestId.randomUUID().toString(),
|
||||
val patientId: String,
|
||||
val pid: String,
|
||||
val fingerprint: String,
|
||||
val uuid: RequestId = randomRequestId(),
|
||||
val patientPseudonym: PatientPseudonym,
|
||||
val pid: PatientId,
|
||||
@Column("fingerprint")
|
||||
val fingerprint: Fingerprint,
|
||||
val type: RequestType,
|
||||
var status: RequestStatus,
|
||||
var processedAt: Instant = Instant.now(),
|
||||
@Embedded.Nullable var report: Report? = null
|
||||
)
|
||||
) {
|
||||
constructor(
|
||||
uuid: RequestId,
|
||||
patientPseudonym: PatientPseudonym,
|
||||
pid: PatientId,
|
||||
fingerprint: Fingerprint,
|
||||
type: RequestType,
|
||||
status: RequestStatus
|
||||
) :
|
||||
this(null, uuid, patientPseudonym, pid, fingerprint, type, status, Instant.now())
|
||||
|
||||
constructor(
|
||||
uuid: RequestId,
|
||||
patientPseudonym: PatientPseudonym,
|
||||
pid: PatientId,
|
||||
fingerprint: Fingerprint,
|
||||
type: RequestType,
|
||||
status: RequestStatus,
|
||||
processedAt: Instant
|
||||
) :
|
||||
this(null, uuid, patientPseudonym, pid, fingerprint, type, status, processedAt)
|
||||
}
|
||||
|
||||
@JvmRecord
|
||||
data class Report(
|
||||
val description: String,
|
||||
val dataQualityReport: String = ""
|
||||
)
|
||||
|
||||
@JvmRecord
|
||||
data class CountedState(
|
||||
val count: Int,
|
||||
val status: RequestStatus,
|
||||
@ -57,17 +81,17 @@ data class CountedState(
|
||||
|
||||
interface RequestRepository : CrudRepository<Request, Long>, PagingAndSortingRepository<Request, Long> {
|
||||
|
||||
fun findAllByPatientIdOrderByProcessedAtDesc(patientId: String): List<Request>
|
||||
fun findAllByPatientPseudonymOrderByProcessedAtDesc(patientId: PatientPseudonym): List<Request>
|
||||
|
||||
fun findByUuidEquals(uuid: String): Optional<Request>
|
||||
fun findByUuidEquals(uuid: RequestId): Optional<Request>
|
||||
|
||||
fun findRequestByPatientId(patientId: String, pageable: Pageable): Page<Request>
|
||||
fun findRequestByPatientPseudonym(patientPseudonym: PatientPseudonym, pageable: Pageable): Page<Request>
|
||||
|
||||
@Query("SELECT count(*) AS count, status FROM request WHERE type = 'MTB_FILE' GROUP BY status ORDER BY status, count DESC;")
|
||||
fun countStates(): List<CountedState>
|
||||
|
||||
@Query("SELECT count(*) AS count, status FROM (" +
|
||||
"SELECT status, rank() OVER (PARTITION BY patient_id ORDER BY processed_at DESC) AS rank FROM request " +
|
||||
"SELECT status, rank() OVER (PARTITION BY patient_pseudonym ORDER BY processed_at DESC) AS rank FROM request " +
|
||||
"WHERE type = 'MTB_FILE' AND status NOT IN ('DUPLICATION') " +
|
||||
") rank WHERE rank = 1 GROUP BY status ORDER BY status, count DESC;")
|
||||
fun findPatientUniqueStates(): List<CountedState>
|
||||
@ -76,7 +100,7 @@ interface RequestRepository : CrudRepository<Request, Long>, PagingAndSortingRep
|
||||
fun countDeleteStates(): List<CountedState>
|
||||
|
||||
@Query("SELECT count(*) AS count, status FROM (" +
|
||||
"SELECT status, rank() OVER (PARTITION BY patient_id ORDER BY processed_at DESC) AS rank FROM request " +
|
||||
"SELECT status, rank() OVER (PARTITION BY patient_pseudonym ORDER BY processed_at DESC) AS rank FROM request " +
|
||||
"WHERE type = 'DELETE'" +
|
||||
") rank WHERE rank = 1 GROUP BY status ORDER BY status, count DESC;")
|
||||
fun findPatientUniqueDeleteStates(): List<CountedState>
|
||||
|
@ -1,7 +1,7 @@
|
||||
/*
|
||||
* This file is part of ETL-Processor
|
||||
*
|
||||
* Copyright (c) 2024 Comprehensive Cancer Center Mainfranken, Datenintegrationszentrum Philipps-Universität Marburg and Contributors
|
||||
* Copyright (c) 2025 Comprehensive Cancer Center Mainfranken, Datenintegrationszentrum Philipps-Universität Marburg and Contributors
|
||||
*
|
||||
* This program is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
@ -22,9 +22,12 @@ package dev.dnpm.etl.processor.output
|
||||
import com.fasterxml.jackson.databind.ObjectMapper
|
||||
import de.ukw.ccc.bwhc.dto.Consent
|
||||
import de.ukw.ccc.bwhc.dto.MtbFile
|
||||
import dev.dnpm.etl.processor.CustomMediaType
|
||||
import dev.dnpm.etl.processor.config.KafkaProperties
|
||||
import dev.dnpm.etl.processor.monitoring.RequestStatus
|
||||
import org.apache.kafka.clients.producer.ProducerRecord
|
||||
import org.slf4j.LoggerFactory
|
||||
import org.springframework.http.MediaType
|
||||
import org.springframework.kafka.core.KafkaTemplate
|
||||
import org.springframework.retry.support.RetryTemplate
|
||||
|
||||
@ -37,14 +40,20 @@ class KafkaMtbFileSender(
|
||||
|
||||
private val logger = LoggerFactory.getLogger(KafkaMtbFileSender::class.java)
|
||||
|
||||
override fun send(request: MtbFileSender.MtbFileRequest): MtbFileSender.Response {
|
||||
override fun <T> send(request: MtbFileRequest<T>): MtbFileSender.Response {
|
||||
return try {
|
||||
return retryTemplate.execute<MtbFileSender.Response, Exception> {
|
||||
val result = kafkaTemplate.send(
|
||||
kafkaProperties.topic,
|
||||
key(request),
|
||||
objectMapper.writeValueAsString(Data(request.requestId, request.mtbFile))
|
||||
)
|
||||
val record =
|
||||
ProducerRecord(kafkaProperties.outputTopic, key(request), objectMapper.writeValueAsString(request))
|
||||
when (request) {
|
||||
is BwhcV1MtbFileRequest -> record.headers()
|
||||
.add("contentType", MediaType.APPLICATION_JSON_VALUE.toByteArray())
|
||||
|
||||
is DnpmV2MtbFileRequest -> record.headers()
|
||||
.add("contentType", CustomMediaType.APPLICATION_VND_DNPM_V2_MTB_JSON_VALUE.toByteArray())
|
||||
}
|
||||
|
||||
val result = kafkaTemplate.send(record)
|
||||
if (result.get() != null) {
|
||||
logger.debug("Sent file via KafkaMtbFileSender")
|
||||
MtbFileSender.Response(RequestStatus.UNKNOWN)
|
||||
@ -58,11 +67,11 @@ class KafkaMtbFileSender(
|
||||
}
|
||||
}
|
||||
|
||||
override fun send(request: MtbFileSender.DeleteRequest): MtbFileSender.Response {
|
||||
override fun send(request: DeleteRequest): MtbFileSender.Response {
|
||||
val dummyMtbFile = MtbFile.builder()
|
||||
.withConsent(
|
||||
Consent.builder()
|
||||
.withPatient(request.patientId)
|
||||
.withPatient(request.patientId.value)
|
||||
.withStatus(Consent.Status.REJECTED)
|
||||
.build()
|
||||
)
|
||||
@ -70,12 +79,15 @@ class KafkaMtbFileSender(
|
||||
|
||||
return try {
|
||||
return retryTemplate.execute<MtbFileSender.Response, Exception> {
|
||||
val result = kafkaTemplate.send(
|
||||
kafkaProperties.topic,
|
||||
val record =
|
||||
ProducerRecord(
|
||||
kafkaProperties.outputTopic,
|
||||
key(request),
|
||||
objectMapper.writeValueAsString(Data(request.requestId, dummyMtbFile))
|
||||
// Always use old BwhcV1FileRequest with Consent REJECT
|
||||
objectMapper.writeValueAsString(BwhcV1MtbFileRequest(request.requestId, dummyMtbFile))
|
||||
)
|
||||
|
||||
val result = kafkaTemplate.send(record)
|
||||
if (result.get() != null) {
|
||||
logger.debug("Sent deletion request via KafkaMtbFileSender")
|
||||
MtbFileSender.Response(RequestStatus.UNKNOWN)
|
||||
@ -90,16 +102,15 @@ class KafkaMtbFileSender(
|
||||
}
|
||||
|
||||
override fun endpoint(): String {
|
||||
return "${this.kafkaProperties.servers} (${this.kafkaProperties.topic}/${this.kafkaProperties.responseTopic})"
|
||||
return "${this.kafkaProperties.servers} (${this.kafkaProperties.outputTopic}/${this.kafkaProperties.outputResponseTopic})"
|
||||
}
|
||||
|
||||
private fun key(request: MtbFileSender.MtbFileRequest): String {
|
||||
return "{\"pid\": \"${request.mtbFile.patient.id}\"}"
|
||||
private fun key(request: MtbRequest): String {
|
||||
return when (request) {
|
||||
is BwhcV1MtbFileRequest -> "{\"pid\": \"${request.content.patient.id}\"}"
|
||||
is DnpmV2MtbFileRequest -> "{\"pid\": \"${request.content.patient.id}\"}"
|
||||
is DeleteRequest -> "{\"pid\": \"${request.patientId.value}\"}"
|
||||
else -> throw IllegalArgumentException("Unsupported request type: ${request::class.simpleName}")
|
||||
}
|
||||
|
||||
private fun key(request: MtbFileSender.DeleteRequest): String {
|
||||
return "{\"pid\": \"${request.patientId}\"}"
|
||||
}
|
||||
|
||||
data class Data(val requestId: String, val content: MtbFile)
|
||||
}
|
@ -19,23 +19,17 @@
|
||||
|
||||
package dev.dnpm.etl.processor.output
|
||||
|
||||
import de.ukw.ccc.bwhc.dto.MtbFile
|
||||
import dev.dnpm.etl.processor.monitoring.RequestStatus
|
||||
import org.springframework.http.HttpStatusCode
|
||||
|
||||
interface MtbFileSender {
|
||||
fun send(request: MtbFileRequest): Response
|
||||
fun <T> send(request: MtbFileRequest<T>): Response
|
||||
|
||||
fun send(request: DeleteRequest): Response
|
||||
|
||||
fun endpoint(): String
|
||||
|
||||
data class Response(val status: RequestStatus, val body: String = "")
|
||||
|
||||
data class MtbFileRequest(val requestId: String, val mtbFile: MtbFile)
|
||||
|
||||
data class DeleteRequest(val requestId: String, val patientId: String)
|
||||
|
||||
}
|
||||
|
||||
fun Int.asRequestStatus(): RequestStatus {
|
||||
|
59
src/main/kotlin/dev/dnpm/etl/processor/output/MtbRequest.kt
Normal file
59
src/main/kotlin/dev/dnpm/etl/processor/output/MtbRequest.kt
Normal file
@ -0,0 +1,59 @@
|
||||
/*
|
||||
* This file is part of ETL-Processor
|
||||
*
|
||||
* Copyright (c) 2025 Comprehensive Cancer Center Mainfranken, Datenintegrationszentrum Philipps-Universität Marburg and Contributors
|
||||
*
|
||||
* This program is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
* by the Free Software Foundation, either version 3 of the License, or
|
||||
* (at your option) any later version.
|
||||
*
|
||||
* This program is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
* GNU Affero General Public License for more details.
|
||||
*
|
||||
* You should have received a copy of the GNU Affero General Public License
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
*/
|
||||
|
||||
package dev.dnpm.etl.processor.output
|
||||
|
||||
import de.ukw.ccc.bwhc.dto.MtbFile
|
||||
import dev.dnpm.etl.processor.PatientPseudonym
|
||||
import dev.dnpm.etl.processor.RequestId
|
||||
import dev.pcvolkmer.mv64e.mtb.Mtb
|
||||
|
||||
interface MtbRequest {
|
||||
val requestId: RequestId
|
||||
}
|
||||
|
||||
sealed interface MtbFileRequest<out T> : MtbRequest {
|
||||
override val requestId: RequestId
|
||||
val content: T
|
||||
|
||||
fun patientPseudonym(): PatientPseudonym
|
||||
}
|
||||
|
||||
data class BwhcV1MtbFileRequest(
|
||||
override val requestId: RequestId,
|
||||
override val content: MtbFile
|
||||
) : MtbFileRequest<MtbFile> {
|
||||
override fun patientPseudonym(): PatientPseudonym {
|
||||
return PatientPseudonym(content.patient.id)
|
||||
}
|
||||
}
|
||||
|
||||
data class DnpmV2MtbFileRequest(
|
||||
override val requestId: RequestId,
|
||||
override val content: Mtb
|
||||
) : MtbFileRequest<Mtb> {
|
||||
override fun patientPseudonym(): PatientPseudonym {
|
||||
return PatientPseudonym(content.patient.id)
|
||||
}
|
||||
}
|
||||
|
||||
data class DeleteRequest(
|
||||
override val requestId: RequestId,
|
||||
val patientId: PatientPseudonym
|
||||
) : MtbRequest
|
@ -0,0 +1,51 @@
|
||||
/*
|
||||
* This file is part of ETL-Processor
|
||||
*
|
||||
* Copyright (c) 2025 Comprehensive Cancer Center Mainfranken, Datenintegrationszentrum Philipps-Universität Marburg and Contributors
|
||||
*
|
||||
* This program is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
* by the Free Software Foundation, either version 3 of the License, or
|
||||
* (at your option) any later version.
|
||||
*
|
||||
* This program is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
* GNU Affero General Public License for more details.
|
||||
*
|
||||
* You should have received a copy of the GNU Affero General Public License
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
*/
|
||||
|
||||
package dev.dnpm.etl.processor.output
|
||||
|
||||
import dev.dnpm.etl.processor.PatientPseudonym
|
||||
import dev.dnpm.etl.processor.config.RestTargetProperties
|
||||
import dev.dnpm.etl.processor.monitoring.ReportService
|
||||
import org.springframework.retry.support.RetryTemplate
|
||||
import org.springframework.web.client.RestTemplate
|
||||
import org.springframework.web.util.UriComponentsBuilder
|
||||
|
||||
class RestBwhcMtbFileSender(
|
||||
restTemplate: RestTemplate,
|
||||
private val restTargetProperties: RestTargetProperties,
|
||||
retryTemplate: RetryTemplate,
|
||||
reportService: ReportService,
|
||||
) : RestMtbFileSender(restTemplate, restTargetProperties, retryTemplate, reportService) {
|
||||
|
||||
override fun sendUrl(): String {
|
||||
return UriComponentsBuilder
|
||||
.fromUriString(restTargetProperties.uri.toString())
|
||||
.pathSegment("MTBFile")
|
||||
.toUriString()
|
||||
}
|
||||
|
||||
override fun deleteUrl(patientId: PatientPseudonym): String {
|
||||
return UriComponentsBuilder
|
||||
.fromUriString(restTargetProperties.uri.toString())
|
||||
.pathSegment("Patient")
|
||||
.pathSegment(patientId.value)
|
||||
.toUriString()
|
||||
}
|
||||
|
||||
}
|
@ -0,0 +1,55 @@
|
||||
/*
|
||||
* This file is part of ETL-Processor
|
||||
*
|
||||
* Copyright (c) 2025 Comprehensive Cancer Center Mainfranken, Datenintegrationszentrum Philipps-Universität Marburg and Contributors
|
||||
*
|
||||
* This program is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
* by the Free Software Foundation, either version 3 of the License, or
|
||||
* (at your option) any later version.
|
||||
*
|
||||
* This program is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
* GNU Affero General Public License for more details.
|
||||
*
|
||||
* You should have received a copy of the GNU Affero General Public License
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
*/
|
||||
|
||||
package dev.dnpm.etl.processor.output
|
||||
|
||||
import dev.dnpm.etl.processor.PatientPseudonym
|
||||
import dev.dnpm.etl.processor.config.RestTargetProperties
|
||||
import dev.dnpm.etl.processor.monitoring.ReportService
|
||||
import org.springframework.retry.support.RetryTemplate
|
||||
import org.springframework.web.client.RestTemplate
|
||||
import org.springframework.web.util.UriComponentsBuilder
|
||||
|
||||
class RestDipMtbFileSender(
|
||||
restTemplate: RestTemplate,
|
||||
private val restTargetProperties: RestTargetProperties,
|
||||
retryTemplate: RetryTemplate,
|
||||
reportService: ReportService
|
||||
) : RestMtbFileSender(restTemplate, restTargetProperties, retryTemplate, reportService) {
|
||||
|
||||
override fun sendUrl(): String {
|
||||
return UriComponentsBuilder
|
||||
.fromUriString(restTargetProperties.uri.toString())
|
||||
.pathSegment("mtb")
|
||||
.pathSegment("etl")
|
||||
.pathSegment("patient-record")
|
||||
.toUriString()
|
||||
}
|
||||
|
||||
override fun deleteUrl(patientId: PatientPseudonym): String {
|
||||
return UriComponentsBuilder
|
||||
.fromUriString(restTargetProperties.uri.toString())
|
||||
.pathSegment("mtb")
|
||||
.pathSegment("etl")
|
||||
.pathSegment("patient")
|
||||
.pathSegment(patientId.value)
|
||||
.toUriString()
|
||||
}
|
||||
|
||||
}
|
@ -1,7 +1,7 @@
|
||||
/*
|
||||
* This file is part of ETL-Processor
|
||||
*
|
||||
* Copyright (c) 2024 Comprehensive Cancer Center Mainfranken, Datenintegrationszentrum Philipps-Universität Marburg and Contributors
|
||||
* Copyright (c) 2025 Comprehensive Cancer Center Mainfranken, Datenintegrationszentrum Philipps-Universität Marburg and Contributors
|
||||
*
|
||||
* This program is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
@ -19,62 +19,71 @@
|
||||
|
||||
package dev.dnpm.etl.processor.output
|
||||
|
||||
import dev.dnpm.etl.processor.CustomMediaType
|
||||
import dev.dnpm.etl.processor.PatientPseudonym
|
||||
import dev.dnpm.etl.processor.config.RestTargetProperties
|
||||
import dev.dnpm.etl.processor.monitoring.ReportService
|
||||
import dev.dnpm.etl.processor.monitoring.RequestStatus
|
||||
import dev.dnpm.etl.processor.monitoring.asRequestStatus
|
||||
import org.slf4j.LoggerFactory
|
||||
import org.springframework.http.HttpEntity
|
||||
import org.springframework.http.HttpHeaders
|
||||
import org.springframework.http.MediaType
|
||||
import org.springframework.retry.support.RetryTemplate
|
||||
import org.springframework.web.client.RestClientException
|
||||
import org.springframework.web.client.RestClientResponseException
|
||||
import org.springframework.web.client.RestTemplate
|
||||
|
||||
class RestMtbFileSender(
|
||||
abstract class RestMtbFileSender(
|
||||
private val restTemplate: RestTemplate,
|
||||
private val restTargetProperties: RestTargetProperties,
|
||||
private val retryTemplate: RetryTemplate
|
||||
private val retryTemplate: RetryTemplate,
|
||||
private val reportService: ReportService
|
||||
) : MtbFileSender {
|
||||
|
||||
private val logger = LoggerFactory.getLogger(RestMtbFileSender::class.java)
|
||||
|
||||
override fun send(request: MtbFileSender.MtbFileRequest): MtbFileSender.Response {
|
||||
abstract fun sendUrl(): String
|
||||
|
||||
abstract fun deleteUrl(patientId: PatientPseudonym): String
|
||||
|
||||
override fun <T> send(request: MtbFileRequest<T>): MtbFileSender.Response {
|
||||
try {
|
||||
return retryTemplate.execute<MtbFileSender.Response, Exception> {
|
||||
val headers = HttpHeaders()
|
||||
headers.contentType = MediaType.APPLICATION_JSON
|
||||
val entityReq = HttpEntity(request.mtbFile, headers)
|
||||
val headers = getHttpHeaders(request)
|
||||
val entityReq = HttpEntity(request.content, headers)
|
||||
val response = restTemplate.postForEntity(
|
||||
"${restTargetProperties.uri}/MTBFile",
|
||||
sendUrl(),
|
||||
entityReq,
|
||||
String::class.java
|
||||
)
|
||||
if (!response.statusCode.is2xxSuccessful) {
|
||||
logger.warn("Error sending to remote system: {}", response.body)
|
||||
return@execute MtbFileSender.Response(
|
||||
response.statusCode.asRequestStatus(),
|
||||
reportService.deserialize(response.body).asRequestStatus(),
|
||||
"Status-Code: ${response.statusCode.value()}"
|
||||
)
|
||||
}
|
||||
logger.debug("Sent file via RestMtbFileSender")
|
||||
return@execute MtbFileSender.Response(response.statusCode.asRequestStatus(), response.body.orEmpty())
|
||||
return@execute MtbFileSender.Response(reportService.deserialize(response.body).asRequestStatus(), response.body.orEmpty())
|
||||
}
|
||||
} catch (e: IllegalArgumentException) {
|
||||
logger.error("Not a valid URI to export to: '{}'", restTargetProperties.uri!!)
|
||||
} catch (e: RestClientException) {
|
||||
} catch (e: RestClientResponseException) {
|
||||
logger.info(restTargetProperties.uri!!.toString())
|
||||
logger.error("Cannot send data to remote system", e)
|
||||
logger.error("Request data not accepted by remote system", e)
|
||||
return MtbFileSender.Response(reportService.deserialize(e.responseBodyAsString).asRequestStatus(), e.responseBodyAsString)
|
||||
}
|
||||
return MtbFileSender.Response(RequestStatus.ERROR, "Sonstiger Fehler bei der Übertragung")
|
||||
}
|
||||
|
||||
override fun send(request: MtbFileSender.DeleteRequest): MtbFileSender.Response {
|
||||
override fun send(request: DeleteRequest): MtbFileSender.Response {
|
||||
try {
|
||||
return retryTemplate.execute<MtbFileSender.Response, Exception> {
|
||||
val headers = HttpHeaders()
|
||||
headers.contentType = MediaType.APPLICATION_JSON
|
||||
val headers = getHttpHeaders(request)
|
||||
val entityReq = HttpEntity(null, headers)
|
||||
restTemplate.delete(
|
||||
"${restTargetProperties.uri}/Patient/${request.patientId}",
|
||||
deleteUrl(request.patientId),
|
||||
entityReq,
|
||||
String::class.java
|
||||
)
|
||||
@ -94,4 +103,22 @@ class RestMtbFileSender(
|
||||
return this.restTargetProperties.uri.orEmpty()
|
||||
}
|
||||
|
||||
private fun getHttpHeaders(request: MtbRequest): HttpHeaders {
|
||||
val username = restTargetProperties.username
|
||||
val password = restTargetProperties.password
|
||||
val headers = HttpHeaders()
|
||||
headers.contentType = when (request) {
|
||||
is BwhcV1MtbFileRequest -> MediaType.APPLICATION_JSON
|
||||
is DnpmV2MtbFileRequest -> CustomMediaType.APPLICATION_VND_DNPM_V2_MTB_JSON
|
||||
else -> MediaType.APPLICATION_JSON
|
||||
}
|
||||
|
||||
if (username.isNullOrBlank() || password.isNullOrBlank()) {
|
||||
return headers
|
||||
}
|
||||
|
||||
headers.setBasicAuth(username, password)
|
||||
return headers
|
||||
}
|
||||
|
||||
}
|
@ -19,6 +19,8 @@
|
||||
|
||||
package dev.dnpm.etl.processor.pseudonym
|
||||
|
||||
import dev.dnpm.etl.processor.PatientId
|
||||
import dev.dnpm.etl.processor.PatientPseudonym
|
||||
import dev.dnpm.etl.processor.config.PseudonymizeConfigProperties
|
||||
|
||||
class PseudonymizeService(
|
||||
@ -26,10 +28,10 @@ class PseudonymizeService(
|
||||
private val configProperties: PseudonymizeConfigProperties
|
||||
) {
|
||||
|
||||
fun patientPseudonym(patientId: String): String {
|
||||
fun patientPseudonym(patientId: PatientId): PatientPseudonym {
|
||||
return when (generator) {
|
||||
is GpasPseudonymGenerator -> generator.generate(patientId)
|
||||
else -> "${configProperties.prefix}_${generator.generate(patientId)}"
|
||||
is GpasPseudonymGenerator -> PatientPseudonym(generator.generate(patientId.value))
|
||||
else -> PatientPseudonym("${configProperties.prefix}_${generator.generate(patientId.value)}")
|
||||
}
|
||||
}
|
||||
|
||||
|
@ -20,16 +20,17 @@
|
||||
package dev.dnpm.etl.processor.pseudonym
|
||||
|
||||
import de.ukw.ccc.bwhc.dto.MtbFile
|
||||
import dev.dnpm.etl.processor.PatientId
|
||||
import dev.pcvolkmer.mv64e.mtb.Mtb
|
||||
import org.apache.commons.codec.digest.DigestUtils
|
||||
|
||||
/** Replaces patient ID with generated patient pseudonym
|
||||
*
|
||||
* @param pseudonymizeService The pseudonymizeService to be used
|
||||
*
|
||||
* @return The MTB file containing patient pseudonymes
|
||||
*/
|
||||
infix fun MtbFile.pseudonymizeWith(pseudonymizeService: PseudonymizeService) {
|
||||
val patientPseudonym = pseudonymizeService.patientPseudonym(this.patient.id)
|
||||
val patientPseudonym = pseudonymizeService.patientPseudonym(PatientId(this.patient.id)).value
|
||||
|
||||
this.episode?.patient = patientPseudonym
|
||||
this.carePlans?.forEach { it.patient = patientPseudonym }
|
||||
@ -48,7 +49,11 @@ infix fun MtbFile.pseudonymizeWith(pseudonymizeService: PseudonymizeService) {
|
||||
}
|
||||
this.lastGuidelineTherapies?.forEach { it.patient = patientPseudonym }
|
||||
this.molecularPathologyFindings?.forEach { it.patient = patientPseudonym }
|
||||
this.molecularTherapies?.forEach { molecularTherapy -> molecularTherapy.history.forEach { it.patient = patientPseudonym } }
|
||||
this.molecularTherapies?.forEach { molecularTherapy ->
|
||||
molecularTherapy.history.forEach {
|
||||
it.patient = patientPseudonym
|
||||
}
|
||||
}
|
||||
this.ngsReports?.forEach { it.patient = patientPseudonym }
|
||||
this.previousGuidelineTherapies?.forEach { it.patient = patientPseudonym }
|
||||
this.rebiopsyRequests?.forEach { it.patient = patientPseudonym }
|
||||
@ -62,7 +67,6 @@ infix fun MtbFile.pseudonymizeWith(pseudonymizeService: PseudonymizeService) {
|
||||
* Creates new hash of content IDs with given prefix except for patient IDs
|
||||
*
|
||||
* @param pseudonymizeService The pseudonymizeService to be used
|
||||
*
|
||||
* @return The MTB file containing rehashed content IDs
|
||||
*/
|
||||
infix fun MtbFile.anonymizeContentWith(pseudonymizeService: PseudonymizeService) {
|
||||
@ -223,3 +227,89 @@ infix fun MtbFile.anonymizeContentWith(pseudonymizeService: PseudonymizeService)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/** Replaces patient ID with generated patient pseudonym
|
||||
*
|
||||
* @since 0.11.0
|
||||
*
|
||||
* @param pseudonymizeService The pseudonymizeService to be used
|
||||
* @return The MTB file containing patient pseudonymes
|
||||
*/
|
||||
infix fun Mtb.pseudonymizeWith(pseudonymizeService: PseudonymizeService) {
|
||||
val patientPseudonym = pseudonymizeService.patientPseudonym(PatientId(this.patient.id)).value
|
||||
|
||||
this.episodesOfCare?.forEach { it.patient.id = patientPseudonym }
|
||||
this.carePlans?.forEach {
|
||||
it.patient.id = patientPseudonym
|
||||
it.rebiopsyRequests?.forEach { it.patient.id = patientPseudonym }
|
||||
it.histologyReevaluationRequests?.forEach { it.patient.id = patientPseudonym }
|
||||
it.medicationRecommendations.forEach { it.patient.id = patientPseudonym }
|
||||
it.studyEnrollmentRecommendations?.forEach { it.patient.id = patientPseudonym }
|
||||
it.procedureRecommendations?.forEach { it.patient.id = patientPseudonym }
|
||||
it.geneticCounselingRecommendation.patient.id = patientPseudonym
|
||||
}
|
||||
this.diagnoses?.forEach { it.patient.id = patientPseudonym }
|
||||
this.guidelineTherapies?.forEach { it.patient.id = patientPseudonym }
|
||||
this.guidelineProcedures?.forEach { it.patient.id = patientPseudonym }
|
||||
this.patient.id = patientPseudonym
|
||||
this.claims?.forEach { it.patient.id = patientPseudonym }
|
||||
this.claimResponses?.forEach { it.patient.id = patientPseudonym }
|
||||
this.diagnoses?.forEach { it.patient.id = patientPseudonym }
|
||||
this.histologyReports?.forEach {
|
||||
it.patient.id = patientPseudonym
|
||||
it.results.tumorMorphology?.patient?.id = patientPseudonym
|
||||
it.results.tumorCellContent?.patient?.id = patientPseudonym
|
||||
}
|
||||
this.ngsReports?.forEach {
|
||||
it.patient.id = patientPseudonym
|
||||
it.results.simpleVariants?.forEach { it.patient.id = patientPseudonym }
|
||||
it.results.copyNumberVariants?.forEach { it.patient.id = patientPseudonym }
|
||||
it.results.dnaFusions?.forEach { it.patient.id = patientPseudonym }
|
||||
it.results.rnaFusions?.forEach { it.patient.id = patientPseudonym }
|
||||
it.results.tumorCellContent?.patient?.id = patientPseudonym
|
||||
it.results.brcaness?.patient?.id = patientPseudonym
|
||||
it.results.tmb?.patient?.id = patientPseudonym
|
||||
it.results.hrdScore?.patient?.id = patientPseudonym
|
||||
}
|
||||
this.ihcReports?.forEach {
|
||||
it.patient.id = patientPseudonym
|
||||
it.results.msiMmr?.forEach { it.patient.id = patientPseudonym }
|
||||
it.results.proteinExpression?.forEach { it.patient.id = patientPseudonym }
|
||||
}
|
||||
this.responses?.forEach { it.patient.id = patientPseudonym }
|
||||
this.specimens?.forEach { it.patient.id = patientPseudonym }
|
||||
this.priorDiagnosticReports?.forEach { it.patient.id = patientPseudonym }
|
||||
this.performanceStatus.forEach { it.patient.id = patientPseudonym }
|
||||
this.systemicTherapies.forEach {
|
||||
it.history?.forEach {
|
||||
it.patient.id = patientPseudonym
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Creates new hash of content IDs with given prefix except for patient IDs
|
||||
*
|
||||
* @since 0.11.0
|
||||
*
|
||||
* @param pseudonymizeService The pseudonymizeService to be used
|
||||
* @return The MTB file containing rehashed content IDs
|
||||
*/
|
||||
infix fun Mtb.anonymizeContentWith(pseudonymizeService: PseudonymizeService) {
|
||||
val prefix = pseudonymizeService.prefix()
|
||||
|
||||
fun anonymize(id: String): String {
|
||||
val hash = DigestUtils.sha256Hex("$prefix-$id").substring(0, 41).lowercase()
|
||||
return "$prefix$hash"
|
||||
}
|
||||
|
||||
this.episodesOfCare?.forEach {
|
||||
it?.apply {
|
||||
id = id?.let {
|
||||
anonymize(it)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// TODO all other properties
|
||||
}
|
||||
|
@ -17,7 +17,7 @@
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
*/
|
||||
|
||||
package dev.dnpm.etl.processor.services
|
||||
package dev.dnpm.etl.processor.security
|
||||
|
||||
import jakarta.annotation.PostConstruct
|
||||
import org.springframework.data.annotation.Id
|
@ -17,11 +17,8 @@
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
*/
|
||||
|
||||
package dev.dnpm.etl.processor.services
|
||||
package dev.dnpm.etl.processor.security
|
||||
|
||||
import dev.dnpm.etl.processor.security.Role
|
||||
import dev.dnpm.etl.processor.security.UserRole
|
||||
import dev.dnpm.etl.processor.security.UserRoleRepository
|
||||
import org.springframework.data.repository.findByIdOrNull
|
||||
import org.springframework.security.core.session.SessionRegistry
|
||||
import org.springframework.security.oauth2.core.oidc.user.OidcUser
|
@ -1,7 +1,7 @@
|
||||
/*
|
||||
* This file is part of ETL-Processor
|
||||
*
|
||||
* Copyright (c) 2023 Comprehensive Cancer Center Mainfranken, Datenintegrationszentrum Philipps-Universität Marburg and Contributors
|
||||
* Copyright (c) 2025 Comprehensive Cancer Center Mainfranken, Datenintegrationszentrum Philipps-Universität Marburg and Contributors
|
||||
*
|
||||
* This program is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
@ -21,15 +21,17 @@ package dev.dnpm.etl.processor.services
|
||||
|
||||
import com.fasterxml.jackson.databind.ObjectMapper
|
||||
import de.ukw.ccc.bwhc.dto.MtbFile
|
||||
import dev.dnpm.etl.processor.*
|
||||
import dev.dnpm.etl.processor.config.AppConfigProperties
|
||||
import dev.dnpm.etl.processor.monitoring.Report
|
||||
import dev.dnpm.etl.processor.monitoring.Request
|
||||
import dev.dnpm.etl.processor.monitoring.RequestStatus
|
||||
import dev.dnpm.etl.processor.monitoring.RequestType
|
||||
import dev.dnpm.etl.processor.output.MtbFileSender
|
||||
import dev.dnpm.etl.processor.output.*
|
||||
import dev.dnpm.etl.processor.pseudonym.PseudonymizeService
|
||||
import dev.dnpm.etl.processor.pseudonym.anonymizeContentWith
|
||||
import dev.dnpm.etl.processor.pseudonym.pseudonymizeWith
|
||||
import dev.pcvolkmer.mv64e.mtb.Mtb
|
||||
import org.apache.commons.codec.binary.Base32
|
||||
import org.apache.commons.codec.digest.DigestUtils
|
||||
import org.springframework.context.ApplicationEventPublisher
|
||||
@ -49,32 +51,45 @@ class RequestProcessor(
|
||||
) {
|
||||
|
||||
fun processMtbFile(mtbFile: MtbFile) {
|
||||
processMtbFile(mtbFile, UUID.randomUUID().toString())
|
||||
processMtbFile(mtbFile, randomRequestId())
|
||||
}
|
||||
|
||||
fun processMtbFile(mtbFile: MtbFile, requestId: String) {
|
||||
val pid = mtbFile.patient.id
|
||||
|
||||
fun processMtbFile(mtbFile: MtbFile, requestId: RequestId) {
|
||||
val pid = PatientId(mtbFile.patient.id)
|
||||
mtbFile pseudonymizeWith pseudonymizeService
|
||||
mtbFile anonymizeContentWith pseudonymizeService
|
||||
val request = BwhcV1MtbFileRequest(requestId, transformationService.transform(mtbFile))
|
||||
saveAndSend(request, pid)
|
||||
}
|
||||
|
||||
val request = MtbFileSender.MtbFileRequest(requestId, transformationService.transform(mtbFile))
|
||||
fun processMtbFile(mtbFile: Mtb) {
|
||||
processMtbFile(mtbFile, randomRequestId())
|
||||
}
|
||||
|
||||
fun processMtbFile(mtbFile: Mtb, requestId: RequestId) {
|
||||
val pid = PatientId(mtbFile.patient.id)
|
||||
mtbFile pseudonymizeWith pseudonymizeService
|
||||
mtbFile anonymizeContentWith pseudonymizeService
|
||||
val request = DnpmV2MtbFileRequest(requestId, transformationService.transform(mtbFile))
|
||||
saveAndSend(request, pid)
|
||||
}
|
||||
|
||||
private fun <T> saveAndSend(request: MtbFileRequest<T>, pid: PatientId) {
|
||||
requestService.save(
|
||||
Request(
|
||||
uuid = requestId,
|
||||
patientId = request.mtbFile.patient.id,
|
||||
pid = pid,
|
||||
fingerprint = fingerprint(request.mtbFile),
|
||||
status = RequestStatus.UNKNOWN,
|
||||
type = RequestType.MTB_FILE
|
||||
request.requestId,
|
||||
request.patientPseudonym(),
|
||||
pid,
|
||||
fingerprint(request),
|
||||
RequestType.MTB_FILE,
|
||||
RequestStatus.UNKNOWN
|
||||
)
|
||||
)
|
||||
|
||||
if (appConfigProperties.duplicationDetection && isDuplication(mtbFile)) {
|
||||
if (appConfigProperties.duplicationDetection && isDuplication(request)) {
|
||||
applicationEventPublisher.publishEvent(
|
||||
ResponseEvent(
|
||||
requestId,
|
||||
request.requestId,
|
||||
Instant.now(),
|
||||
RequestStatus.DUPLICATION
|
||||
)
|
||||
@ -86,47 +101,52 @@ class RequestProcessor(
|
||||
|
||||
applicationEventPublisher.publishEvent(
|
||||
ResponseEvent(
|
||||
requestId,
|
||||
request.requestId,
|
||||
Instant.now(),
|
||||
responseStatus.status,
|
||||
when (responseStatus.status) {
|
||||
RequestStatus.WARNING -> Optional.of(responseStatus.body)
|
||||
RequestStatus.ERROR, RequestStatus.WARNING -> Optional.of(responseStatus.body)
|
||||
else -> Optional.empty()
|
||||
}
|
||||
)
|
||||
)
|
||||
}
|
||||
|
||||
private fun isDuplication(pseudonymizedMtbFile: MtbFile): Boolean {
|
||||
private fun <T> isDuplication(pseudonymizedMtbFileRequest: MtbFileRequest<T>): Boolean {
|
||||
val patientPseudonym = when (pseudonymizedMtbFileRequest) {
|
||||
is BwhcV1MtbFileRequest -> PatientPseudonym(pseudonymizedMtbFileRequest.content.patient.id)
|
||||
is DnpmV2MtbFileRequest -> PatientPseudonym(pseudonymizedMtbFileRequest.content.patient.id)
|
||||
}
|
||||
|
||||
val lastMtbFileRequestForPatient =
|
||||
requestService.lastMtbFileRequestForPatientPseudonym(pseudonymizedMtbFile.patient.id)
|
||||
val isLastRequestDeletion = requestService.isLastRequestWithKnownStatusDeletion(pseudonymizedMtbFile.patient.id)
|
||||
requestService.lastMtbFileRequestForPatientPseudonym(patientPseudonym)
|
||||
val isLastRequestDeletion = requestService.isLastRequestWithKnownStatusDeletion(patientPseudonym)
|
||||
|
||||
return null != lastMtbFileRequestForPatient
|
||||
&& !isLastRequestDeletion
|
||||
&& lastMtbFileRequestForPatient.fingerprint == fingerprint(pseudonymizedMtbFile)
|
||||
&& lastMtbFileRequestForPatient.fingerprint == fingerprint(pseudonymizedMtbFileRequest)
|
||||
}
|
||||
|
||||
fun processDeletion(patientId: String) {
|
||||
processDeletion(patientId, UUID.randomUUID().toString())
|
||||
fun processDeletion(patientId: PatientId) {
|
||||
processDeletion(patientId, randomRequestId())
|
||||
}
|
||||
|
||||
fun processDeletion(patientId: String, requestId: String) {
|
||||
fun processDeletion(patientId: PatientId, requestId: RequestId) {
|
||||
try {
|
||||
val patientPseudonym = pseudonymizeService.patientPseudonym(patientId)
|
||||
|
||||
requestService.save(
|
||||
Request(
|
||||
uuid = requestId,
|
||||
patientId = patientPseudonym,
|
||||
pid = patientId,
|
||||
fingerprint = fingerprint(patientPseudonym),
|
||||
status = RequestStatus.UNKNOWN,
|
||||
type = RequestType.DELETE
|
||||
requestId,
|
||||
patientPseudonym,
|
||||
patientId,
|
||||
fingerprint(patientPseudonym.value),
|
||||
RequestType.DELETE,
|
||||
RequestStatus.UNKNOWN
|
||||
)
|
||||
)
|
||||
|
||||
val responseStatus = sender.send(MtbFileSender.DeleteRequest(requestId, patientPseudonym))
|
||||
val responseStatus = sender.send(DeleteRequest(requestId, patientPseudonym))
|
||||
|
||||
applicationEventPublisher.publishEvent(
|
||||
ResponseEvent(
|
||||
@ -144,9 +164,9 @@ class RequestProcessor(
|
||||
requestService.save(
|
||||
Request(
|
||||
uuid = requestId,
|
||||
patientId = "???",
|
||||
patientPseudonym = emptyPatientPseudonym(),
|
||||
pid = patientId,
|
||||
fingerprint = "",
|
||||
fingerprint = Fingerprint.empty(),
|
||||
status = RequestStatus.ERROR,
|
||||
type = RequestType.DELETE,
|
||||
report = Report("Fehler bei der Pseudonymisierung")
|
||||
@ -155,14 +175,19 @@ class RequestProcessor(
|
||||
}
|
||||
}
|
||||
|
||||
private fun fingerprint(mtbFile: MtbFile): String {
|
||||
return fingerprint(objectMapper.writeValueAsString(mtbFile))
|
||||
private fun <T> fingerprint(request: MtbFileRequest<T>): Fingerprint {
|
||||
return when (request) {
|
||||
is BwhcV1MtbFileRequest -> fingerprint(objectMapper.writeValueAsString(request.content))
|
||||
is DnpmV2MtbFileRequest -> fingerprint(objectMapper.writeValueAsString(request.content))
|
||||
}
|
||||
}
|
||||
|
||||
private fun fingerprint(s: String): String {
|
||||
return Base32().encodeAsString(DigestUtils.sha256(s))
|
||||
private fun fingerprint(s: String): Fingerprint {
|
||||
return Fingerprint(
|
||||
Base32().encodeAsString(DigestUtils.sha256(s))
|
||||
.replace("=", "")
|
||||
.lowercase()
|
||||
)
|
||||
}
|
||||
|
||||
}
|
@ -19,11 +19,13 @@
|
||||
|
||||
package dev.dnpm.etl.processor.services
|
||||
|
||||
import dev.dnpm.etl.processor.monitoring.Request
|
||||
import dev.dnpm.etl.processor.monitoring.RequestRepository
|
||||
import dev.dnpm.etl.processor.monitoring.RequestStatus
|
||||
import dev.dnpm.etl.processor.monitoring.RequestType
|
||||
import dev.dnpm.etl.processor.PatientPseudonym
|
||||
import dev.dnpm.etl.processor.RequestId
|
||||
import dev.dnpm.etl.processor.monitoring.*
|
||||
import org.springframework.data.domain.Page
|
||||
import org.springframework.data.domain.Pageable
|
||||
import org.springframework.stereotype.Service
|
||||
import java.util.*
|
||||
|
||||
@Service
|
||||
class RequestService(
|
||||
@ -32,15 +34,32 @@ class RequestService(
|
||||
|
||||
fun save(request: Request) = requestRepository.save(request)
|
||||
|
||||
fun allRequestsByPatientPseudonym(patientPseudonym: String) = requestRepository
|
||||
.findAllByPatientIdOrderByProcessedAtDesc(patientPseudonym)
|
||||
fun findAll(): Iterable<Request> = requestRepository.findAll()
|
||||
|
||||
fun lastMtbFileRequestForPatientPseudonym(patientPseudonym: String) =
|
||||
fun findAll(pageable: Pageable): Page<Request> = requestRepository.findAll(pageable)
|
||||
|
||||
fun findByUuid(uuid: RequestId): Optional<Request> =
|
||||
requestRepository.findByUuidEquals(uuid)
|
||||
|
||||
fun findRequestByPatientId(patientPseudonym: PatientPseudonym, pageable: Pageable): Page<Request> = requestRepository.findRequestByPatientPseudonym(patientPseudonym, pageable)
|
||||
|
||||
fun allRequestsByPatientPseudonym(patientPseudonym: PatientPseudonym) = requestRepository
|
||||
.findAllByPatientPseudonymOrderByProcessedAtDesc(patientPseudonym)
|
||||
|
||||
fun lastMtbFileRequestForPatientPseudonym(patientPseudonym: PatientPseudonym) =
|
||||
Companion.lastMtbFileRequestForPatientPseudonym(allRequestsByPatientPseudonym(patientPseudonym))
|
||||
|
||||
fun isLastRequestWithKnownStatusDeletion(patientPseudonym: String) =
|
||||
fun isLastRequestWithKnownStatusDeletion(patientPseudonym: PatientPseudonym) =
|
||||
Companion.isLastRequestWithKnownStatusDeletion(allRequestsByPatientPseudonym(patientPseudonym))
|
||||
|
||||
fun countStates(): Iterable<CountedState> = requestRepository.countStates()
|
||||
|
||||
fun countDeleteStates(): Iterable<CountedState> = requestRepository.countDeleteStates()
|
||||
|
||||
fun findPatientUniqueStates(): List<CountedState> = requestRepository.findPatientUniqueStates()
|
||||
|
||||
fun findPatientUniqueDeleteStates(): List<CountedState> = requestRepository.findPatientUniqueDeleteStates()
|
||||
|
||||
companion object {
|
||||
|
||||
fun lastMtbFileRequestForPatientPseudonym(allRequests: List<Request>) = allRequests
|
||||
|
@ -19,8 +19,8 @@
|
||||
|
||||
package dev.dnpm.etl.processor.services
|
||||
|
||||
import dev.dnpm.etl.processor.RequestId
|
||||
import dev.dnpm.etl.processor.monitoring.Report
|
||||
import dev.dnpm.etl.processor.monitoring.RequestRepository
|
||||
import dev.dnpm.etl.processor.monitoring.RequestStatus
|
||||
import org.slf4j.LoggerFactory
|
||||
import org.springframework.context.event.EventListener
|
||||
@ -31,7 +31,7 @@ import java.util.*
|
||||
|
||||
@Service
|
||||
class ResponseProcessor(
|
||||
private val requestRepository: RequestRepository,
|
||||
private val requestService: RequestService,
|
||||
private val statisticsUpdateProducer: Sinks.Many<Any>
|
||||
) {
|
||||
|
||||
@ -39,7 +39,7 @@ class ResponseProcessor(
|
||||
|
||||
@EventListener(classes = [ResponseEvent::class])
|
||||
fun handleResponseEvent(event: ResponseEvent) {
|
||||
requestRepository.findByUuidEquals(event.requestUuid).ifPresentOrElse({
|
||||
requestService.findByUuid(event.requestUuid).ifPresentOrElse({
|
||||
it.processedAt = event.timestamp
|
||||
it.status = event.status
|
||||
|
||||
@ -76,7 +76,7 @@ class ResponseProcessor(
|
||||
}
|
||||
}
|
||||
|
||||
requestRepository.save(it)
|
||||
requestService.save(it)
|
||||
|
||||
statisticsUpdateProducer.emitNext("", Sinks.EmitFailureHandler.FAIL_FAST)
|
||||
}, {
|
||||
@ -87,7 +87,7 @@ class ResponseProcessor(
|
||||
}
|
||||
|
||||
data class ResponseEvent(
|
||||
val requestUuid: String,
|
||||
val requestUuid: RequestId,
|
||||
val timestamp: Instant,
|
||||
val status: RequestStatus,
|
||||
val body: Optional<String> = Optional.empty()
|
||||
|
@ -23,10 +23,21 @@ import com.fasterxml.jackson.databind.ObjectMapper
|
||||
import com.jayway.jsonpath.JsonPath
|
||||
import com.jayway.jsonpath.PathNotFoundException
|
||||
import de.ukw.ccc.bwhc.dto.MtbFile
|
||||
import dev.pcvolkmer.mv64e.mtb.Mtb
|
||||
|
||||
class TransformationService(private val objectMapper: ObjectMapper, private val transformations: List<Transformation>) {
|
||||
fun transform(mtbFile: MtbFile): MtbFile {
|
||||
var json = objectMapper.writeValueAsString(mtbFile)
|
||||
val json = transform(objectMapper.writeValueAsString(mtbFile))
|
||||
return objectMapper.readValue(json, MtbFile::class.java)
|
||||
}
|
||||
|
||||
fun transform(mtbFile: Mtb): Mtb {
|
||||
val json = transform(objectMapper.writeValueAsString(mtbFile))
|
||||
return objectMapper.readValue(json, Mtb::class.java)
|
||||
}
|
||||
|
||||
private fun transform(content: String): String {
|
||||
var json = content
|
||||
|
||||
transformations.forEach { transformation ->
|
||||
val jsonPath = JsonPath.parse(json)
|
||||
@ -48,7 +59,7 @@ class TransformationService(private val objectMapper: ObjectMapper, private val
|
||||
json = jsonPath.jsonString()
|
||||
}
|
||||
|
||||
return objectMapper.readValue(json, MtbFile::class.java)
|
||||
return json
|
||||
}
|
||||
|
||||
fun getTransformations(): List<Transformation> {
|
||||
|
@ -22,6 +22,7 @@ package dev.dnpm.etl.processor.services.kafka
|
||||
import com.fasterxml.jackson.annotation.JsonAlias
|
||||
import com.fasterxml.jackson.annotation.JsonProperty
|
||||
import com.fasterxml.jackson.databind.ObjectMapper
|
||||
import dev.dnpm.etl.processor.RequestId
|
||||
import dev.dnpm.etl.processor.monitoring.RequestStatus
|
||||
import dev.dnpm.etl.processor.output.asRequestStatus
|
||||
import dev.dnpm.etl.processor.services.ResponseEvent
|
||||
@ -47,7 +48,7 @@ class KafkaResponseProcessor(
|
||||
Optional.empty()
|
||||
}.ifPresentOrElse({ responseBody ->
|
||||
val event = ResponseEvent(
|
||||
responseBody.requestId,
|
||||
RequestId(responseBody.requestId),
|
||||
Instant.ofEpochMilli(data.timestamp()),
|
||||
responseBody.statusCode.asRequestStatus(),
|
||||
when (responseBody.statusCode.asRequestStatus()) {
|
||||
|
63
src/main/kotlin/dev/dnpm/etl/processor/types.kt
Normal file
63
src/main/kotlin/dev/dnpm/etl/processor/types.kt
Normal file
@ -0,0 +1,63 @@
|
||||
/*
|
||||
* This file is part of ETL-Processor
|
||||
*
|
||||
* Copyright (c) 2025 Comprehensive Cancer Center Mainfranken, Datenintegrationszentrum Philipps-Universität Marburg and Contributors
|
||||
*
|
||||
* This program is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
* by the Free Software Foundation, either version 3 of the License, or
|
||||
* (at your option) any later version.
|
||||
*
|
||||
* This program is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
* GNU Affero General Public License for more details.
|
||||
*
|
||||
* You should have received a copy of the GNU Affero General Public License
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
*/
|
||||
|
||||
package dev.dnpm.etl.processor
|
||||
|
||||
import org.springframework.http.MediaType
|
||||
import java.util.*
|
||||
|
||||
class Fingerprint(val value: String) {
|
||||
override fun hashCode() = value.hashCode()
|
||||
|
||||
override fun equals(other: Any?) = other is Fingerprint && other.value == value
|
||||
|
||||
companion object {
|
||||
fun empty() = Fingerprint("")
|
||||
}
|
||||
}
|
||||
|
||||
@JvmInline
|
||||
value class RequestId(val value: String) {
|
||||
|
||||
fun isBlank() = value.isBlank()
|
||||
|
||||
}
|
||||
|
||||
fun randomRequestId() = RequestId(UUID.randomUUID().toString())
|
||||
|
||||
@JvmInline
|
||||
value class PatientId(val value: String)
|
||||
|
||||
@JvmInline
|
||||
value class PatientPseudonym(val value: String)
|
||||
|
||||
fun emptyPatientPseudonym() = PatientPseudonym("")
|
||||
|
||||
/**
|
||||
* Custom MediaTypes
|
||||
*
|
||||
* @since 0.11.0
|
||||
*/
|
||||
object CustomMediaType {
|
||||
val APPLICATION_VND_DNPM_V2_MTB_JSON = MediaType("application", "vnd.dnpm.v2.mtb+json")
|
||||
const val APPLICATION_VND_DNPM_V2_MTB_JSON_VALUE = "application/vnd.dnpm.v2.mtb+json"
|
||||
|
||||
val APPLICATION_VND_DNPM_V2_RD_JSON = MediaType("application", "vnd.dnpm.v2.rd+json")
|
||||
const val APPLICATION_VND_DNPM_V2_RD_JSON_VALUE = "application/vnd.dnpm.v2.rd+json"
|
||||
}
|
@ -27,10 +27,10 @@ import dev.dnpm.etl.processor.output.MtbFileSender
|
||||
import dev.dnpm.etl.processor.pseudonym.Generator
|
||||
import dev.dnpm.etl.processor.security.Role
|
||||
import dev.dnpm.etl.processor.security.UserRole
|
||||
import dev.dnpm.etl.processor.services.Token
|
||||
import dev.dnpm.etl.processor.services.TokenService
|
||||
import dev.dnpm.etl.processor.security.Token
|
||||
import dev.dnpm.etl.processor.security.TokenService
|
||||
import dev.dnpm.etl.processor.services.TransformationService
|
||||
import dev.dnpm.etl.processor.services.UserRoleService
|
||||
import dev.dnpm.etl.processor.security.UserRoleService
|
||||
import org.springframework.beans.factory.annotation.Qualifier
|
||||
import org.springframework.http.MediaType
|
||||
import org.springframework.http.codec.ServerSentEvent
|
||||
@ -56,7 +56,7 @@ class ConfigController(
|
||||
@GetMapping
|
||||
fun index(model: Model): String {
|
||||
val outputConnectionAvailable =
|
||||
connectionCheckServices.filterIsInstance<OutputConnectionCheckService>().first().connectionAvailable()
|
||||
connectionCheckServices.filterIsInstance<OutputConnectionCheckService>().firstOrNull()?.connectionAvailable()
|
||||
|
||||
val gPasConnectionAvailable =
|
||||
connectionCheckServices.filterIsInstance<GPasConnectionCheckService>().firstOrNull()?.connectionAvailable()
|
||||
@ -127,10 +127,11 @@ class ConfigController(
|
||||
} else {
|
||||
model.addAttribute("tokensEnabled", true)
|
||||
val result = tokenService.addToken(name)
|
||||
if (result.isSuccess) {
|
||||
model.addAttribute("newTokenValue", result.getOrDefault(""))
|
||||
result.onSuccess {
|
||||
model.addAttribute("newTokenValue", it)
|
||||
model.addAttribute("success", true)
|
||||
} else {
|
||||
}
|
||||
result.onFailure {
|
||||
model.addAttribute("success", false)
|
||||
}
|
||||
model.addAttribute("tokens", tokenService.findAll())
|
||||
@ -182,6 +183,7 @@ class ConfigController(
|
||||
}
|
||||
|
||||
@GetMapping(path = ["events"], produces = [MediaType.TEXT_EVENT_STREAM_VALUE])
|
||||
@ResponseBody
|
||||
fun events(): Flux<ServerSentEvent<Any>> {
|
||||
return connectionCheckUpdateProducer.asFlux().map {
|
||||
val event = when (it) {
|
||||
|
@ -20,9 +20,10 @@
|
||||
package dev.dnpm.etl.processor.web
|
||||
|
||||
import dev.dnpm.etl.processor.NotFoundException
|
||||
import dev.dnpm.etl.processor.PatientPseudonym
|
||||
import dev.dnpm.etl.processor.RequestId
|
||||
import dev.dnpm.etl.processor.monitoring.ReportService
|
||||
import dev.dnpm.etl.processor.monitoring.RequestId
|
||||
import dev.dnpm.etl.processor.monitoring.RequestRepository
|
||||
import dev.dnpm.etl.processor.services.RequestService
|
||||
import org.springframework.data.domain.Pageable
|
||||
import org.springframework.data.domain.Sort
|
||||
import org.springframework.data.web.PageableDefault
|
||||
@ -35,7 +36,7 @@ import org.springframework.web.bind.annotation.RequestMapping
|
||||
@Controller
|
||||
@RequestMapping(path = ["/"])
|
||||
class HomeController(
|
||||
private val requestRepository: RequestRepository,
|
||||
private val requestService: RequestService,
|
||||
private val reportService: ReportService
|
||||
) {
|
||||
|
||||
@ -44,20 +45,20 @@ class HomeController(
|
||||
@PageableDefault(page = 0, size = 20, sort = ["processedAt"], direction = Sort.Direction.DESC) pageable: Pageable,
|
||||
model: Model
|
||||
): String {
|
||||
val requests = requestRepository.findAll(pageable)
|
||||
val requests = requestService.findAll(pageable)
|
||||
model.addAttribute("requests", requests)
|
||||
|
||||
return "index"
|
||||
}
|
||||
|
||||
@GetMapping(path = ["patient/{patientId}"])
|
||||
@GetMapping(path = ["patient/{patientPseudonym}"])
|
||||
fun byPatient(
|
||||
@PathVariable patientId: String,
|
||||
@PathVariable patientPseudonym: PatientPseudonym,
|
||||
@PageableDefault(page = 0, size = 20, sort = ["processedAt"], direction = Sort.Direction.DESC) pageable: Pageable,
|
||||
model: Model
|
||||
): String {
|
||||
val requests = requestRepository.findRequestByPatientId(patientId, pageable)
|
||||
model.addAttribute("patientId", patientId)
|
||||
val requests = requestService.findRequestByPatientId(patientPseudonym, pageable)
|
||||
model.addAttribute("patientPseudonym", patientPseudonym.value)
|
||||
model.addAttribute("requests", requests)
|
||||
|
||||
return "index"
|
||||
@ -65,7 +66,7 @@ class HomeController(
|
||||
|
||||
@GetMapping(path = ["/report/{id}"])
|
||||
fun report(@PathVariable id: RequestId, model: Model): String {
|
||||
val request = requestRepository.findByUuidEquals(id.toString()).orElse(null) ?: throw NotFoundException()
|
||||
val request = requestService.findByUuid(id).orElse(null) ?: throw NotFoundException()
|
||||
model.addAttribute("request", request)
|
||||
model.addAttribute("issues", reportService.deserialize(request.report?.dataQualityReport))
|
||||
|
||||
|
@ -19,9 +19,9 @@
|
||||
|
||||
package dev.dnpm.etl.processor.web
|
||||
|
||||
import dev.dnpm.etl.processor.monitoring.RequestRepository
|
||||
import dev.dnpm.etl.processor.monitoring.RequestStatus
|
||||
import dev.dnpm.etl.processor.monitoring.RequestType
|
||||
import dev.dnpm.etl.processor.services.RequestService
|
||||
import org.springframework.beans.factory.annotation.Qualifier
|
||||
import org.springframework.http.MediaType
|
||||
import org.springframework.http.codec.ServerSentEvent
|
||||
@ -41,15 +41,15 @@ import java.time.temporal.ChronoUnit
|
||||
class StatisticsRestController(
|
||||
@Qualifier("statisticsUpdateProducer")
|
||||
private val statisticsUpdateProducer: Sinks.Many<Any>,
|
||||
private val requestRepository: RequestRepository
|
||||
private val requestService: RequestService
|
||||
) {
|
||||
|
||||
@GetMapping(path = ["requeststates"])
|
||||
fun requestStates(@RequestParam(required = false, defaultValue = "false") delete: Boolean): List<NameValue> {
|
||||
val states = if (delete) {
|
||||
requestRepository.countDeleteStates()
|
||||
requestService.countDeleteStates()
|
||||
} else {
|
||||
requestRepository.countStates()
|
||||
requestService.countStates()
|
||||
}
|
||||
|
||||
return states
|
||||
@ -79,7 +79,7 @@ class StatisticsRestController(
|
||||
}
|
||||
|
||||
val formatter = DateTimeFormatter.ofPattern("yyyy-MM-dd").withZone(ZoneId.of("Europe/Berlin"))
|
||||
val data = requestRepository.findAll()
|
||||
val data = requestService.findAll()
|
||||
.filter { it.type == requestType }
|
||||
.filter { it.processedAt.isAfter(Instant.now().minus(30, ChronoUnit.DAYS)) }
|
||||
.groupBy { formatter.format(it.processedAt) }
|
||||
@ -115,9 +115,9 @@ class StatisticsRestController(
|
||||
@GetMapping(path = ["requestpatientstates"])
|
||||
fun requestPatientStates(@RequestParam(required = false, defaultValue = "false") delete: Boolean): List<NameValue> {
|
||||
val states = if (delete) {
|
||||
requestRepository.findPatientUniqueDeleteStates()
|
||||
requestService.findPatientUniqueDeleteStates()
|
||||
} else {
|
||||
requestRepository.findPatientUniqueStates()
|
||||
requestService.findPatientUniqueStates()
|
||||
}
|
||||
|
||||
return states.map {
|
||||
|
@ -3,17 +3,34 @@ spring:
|
||||
compose:
|
||||
file: ./dev-compose.yml
|
||||
|
||||
security:
|
||||
oauth2:
|
||||
client:
|
||||
registration:
|
||||
custom:
|
||||
client-name: App-Dev
|
||||
client-id: app-dev
|
||||
client-secret: very-secret-ae3f7a-5a9f-1190
|
||||
scope:
|
||||
- openid
|
||||
provider:
|
||||
custom:
|
||||
issuer-uri: https://dnpm.dev/auth/realms/intern
|
||||
user-name-attribute: name
|
||||
|
||||
app:
|
||||
#rest:
|
||||
# uri: http://localhost:9000/bwhc/etl/api
|
||||
kafka:
|
||||
topic: test
|
||||
response-topic: test_response
|
||||
servers: localhost:9094
|
||||
#security:
|
||||
# admin-user: admin
|
||||
# admin-password: "{noop}very-secret"
|
||||
rest:
|
||||
uri: http://localhost:9000/bwhc/etl/api
|
||||
#kafka:
|
||||
# topic: test
|
||||
# response-topic: test_response
|
||||
# servers: localhost:9094
|
||||
security:
|
||||
admin-user: admin
|
||||
admin-password: "{noop}very-secret"
|
||||
enable-oidc: "true"
|
||||
|
||||
server:
|
||||
port: 8000
|
||||
|
||||
|
||||
|
@ -0,0 +1 @@
|
||||
ALTER TABLE request RENAME COLUMN patient_id TO patient_pseudonym;
|
@ -0,0 +1 @@
|
||||
ALTER TABLE request RENAME COLUMN patient_id TO patient_pseudonym;
|
@ -22,6 +22,10 @@
|
||||
--bg-gray-op: rgba(112, 128, 144, .35);
|
||||
}
|
||||
|
||||
* {
|
||||
font-family: sans-serif;
|
||||
}
|
||||
|
||||
html {
|
||||
background: linear-gradient(-5deg, var(--bg-blue-op), transparent 10em);
|
||||
min-height: 100vh;
|
||||
@ -30,7 +34,6 @@ html {
|
||||
|
||||
body {
|
||||
margin: 0 0 5em 0;
|
||||
font-family: sans-serif;
|
||||
font-size: .8rem;
|
||||
color: var(--text);
|
||||
|
||||
@ -619,6 +622,10 @@ input.inline:focus-visible {
|
||||
text-align: center;
|
||||
}
|
||||
|
||||
.notification.info {
|
||||
color: var(--bg-blue);
|
||||
}
|
||||
|
||||
.notification.success {
|
||||
color: var(--bg-green);
|
||||
}
|
||||
@ -643,14 +650,16 @@ input.inline:focus-visible {
|
||||
|
||||
.tab:hover,
|
||||
.tab.active {
|
||||
background: var(--table-border);
|
||||
background: var(--bg-gray);
|
||||
color: white;
|
||||
}
|
||||
|
||||
.tabcontent {
|
||||
border: 1px solid var(--table-border);
|
||||
border: 2px solid var(--bg-gray);
|
||||
border-radius: 0 .5em .5em .5em;
|
||||
display: none;
|
||||
padding: 1em;
|
||||
background: white;
|
||||
}
|
||||
|
||||
.tabcontent.active {
|
||||
@ -689,3 +698,13 @@ a.reload {
|
||||
padding: 1em;
|
||||
background: var(--bg-red-op);
|
||||
}
|
||||
|
||||
.issue-message {
|
||||
font-family: monospace;
|
||||
font-weight: bolder;
|
||||
}
|
||||
|
||||
.issue-path {
|
||||
font-family: monospace;
|
||||
line-height: 1rem;
|
||||
}
|
@ -2,15 +2,20 @@
|
||||
<h2><span>🟦</span> gPAS nicht konfiguriert - Patienten-IDs werden intern anonymisiert</h2>
|
||||
</th:block>
|
||||
<th:block th:if="${gPasConnectionAvailable != null}">
|
||||
<h2><span th:if="${gPasConnectionAvailable}">✅</span><span th:if="${not(gPasConnectionAvailable)}">⚡</span> Verbindung zu gPAS</h2>
|
||||
<h2><span th:if="${gPasConnectionAvailable.available}">✅</span><span th:if="${not(gPasConnectionAvailable.available)}">⚡</span> Verbindung zu gPAS</h2>
|
||||
<div>
|
||||
Die Verbindung ist aktuell
|
||||
<strong th:if="${gPasConnectionAvailable}" style="color: green">verfügbar.</strong>
|
||||
<strong th:if="${not(gPasConnectionAvailable)}" style="color: red">nicht verfügbar.</strong>
|
||||
Stand: <time style="font-weight: bold" th:datetime="${#temporals.formatISO(gPasConnectionAvailable.timestamp)}" th:text="${#temporals.formatISO(gPasConnectionAvailable.timestamp)}"></time>
|
||||
|
|
||||
Letzte Änderung: <time style="font-weight: bold" th:datetime="${#temporals.formatISO(gPasConnectionAvailable.lastChange)}" th:text="${#temporals.formatISO(gPasConnectionAvailable.lastChange)}"></time>
|
||||
</div>
|
||||
<div>
|
||||
<span>Die Verbindung ist aktuell</span>
|
||||
<strong th:if="${gPasConnectionAvailable.available}" style="color: green">verfügbar.</strong>
|
||||
<strong th:if="${not(gPasConnectionAvailable.available)}" style="color: red">nicht verfügbar.</strong>
|
||||
</div>
|
||||
<div class="connection-display border">
|
||||
<img th:src="@{/server.png}" alt="ETL-Processor" />
|
||||
<span class="connection" th:classappend="${gPasConnectionAvailable ? 'available' : ''}"></span>
|
||||
<span class="connection" th:classappend="${gPasConnectionAvailable.available ? 'available' : ''}"></span>
|
||||
<img th:src="@{/server.png}" alt="gPAS" />
|
||||
<span>ETL-Processor</span>
|
||||
<span></span>
|
||||
|
@ -1,16 +1,27 @@
|
||||
<h2><span th:if="${outputConnectionAvailable}">✅</span><span th:if="${not(outputConnectionAvailable)}">⚡</span> MTB-File Verbindung</h2>
|
||||
<th:block th:if="${outputConnectionAvailable == null}">
|
||||
<h2><span>🟦</span> Keine Ausgabenkonfiguration</h2>
|
||||
</th:block>
|
||||
<th:block th:if="${outputConnectionAvailable != null}">
|
||||
<h2><span th:if="${outputConnectionAvailable.available}">✅</span><span th:if="${not(outputConnectionAvailable.available)}">⚡</span> MTB-File Verbindung</h2>
|
||||
<div>
|
||||
Stand: <time style="font-weight: bold" th:datetime="${#temporals.formatISO(outputConnectionAvailable.timestamp)}" th:text="${#temporals.formatISO(outputConnectionAvailable.timestamp)}"></time>
|
||||
|
|
||||
Letzte Änderung: <time style="font-weight: bold" th:datetime="${#temporals.formatISO(outputConnectionAvailable.lastChange)}" th:text="${#temporals.formatISO(outputConnectionAvailable.lastChange)}"></time>
|
||||
</div>
|
||||
<div>
|
||||
Verbindung über <code>[[ ${mtbFileSender} ]]</code>. Die Verbindung ist aktuell
|
||||
<strong th:if="${outputConnectionAvailable}" style="color: green">verfügbar.</strong>
|
||||
<strong th:if="${not(outputConnectionAvailable)}" style="color: red">nicht verfügbar.</strong>
|
||||
<strong th:if="${outputConnectionAvailable.available}" style="color: green">verfügbar.</strong>
|
||||
<strong th:if="${not(outputConnectionAvailable.available)}" style="color: red">nicht verfügbar.</strong>
|
||||
</div>
|
||||
<div class="connection-display border">
|
||||
<img th:src="@{/server.png}" alt="ETL-Processor" />
|
||||
<span class="connection" th:classappend="${outputConnectionAvailable ? 'available' : ''}"></span>
|
||||
<span class="connection" th:classappend="${outputConnectionAvailable.available ? 'available' : ''}"></span>
|
||||
<img th:if="${mtbFileSender.startsWith('Rest')}" th:src="@{/server.png}" alt="bwHC-Backend" />
|
||||
<img th:if="${mtbFileSender.startsWith('Kafka')}" th:src="@{/kafka.png}" alt="Kafka-Broker" />
|
||||
<span>ETL-Processor</span>
|
||||
<span></span>
|
||||
<span th:if="${mtbFileSender.startsWith('Rest')}">bwHC-Backend</span>
|
||||
<span th:if="${mtbFileSender.startsWith('RestBwhc')}">bwHC-Backend</span>
|
||||
<span th:if="${mtbFileSender.startsWith('RestDip')}">DNPM:DIP-Backend</span>
|
||||
<span th:if="${mtbFileSender.startsWith('Kafka')}">Kafka-Broker</span>
|
||||
</div>
|
||||
</th:block>
|
@ -12,26 +12,30 @@
|
||||
<h1>Alle Anfragen<a id="reload-notify" class="reload" title="Neue Anfragen" th:href="@{/}">⟳</a></h1>
|
||||
|
||||
<div>
|
||||
<h2 th:if="${patientId != null}">
|
||||
Betreffend Patienten-Pseudonym <span class="monospace" th:text="${patientId}">***</span>
|
||||
<a class="btn btn-blue" th:if="${patientId != null}" th:href="@{/}">Alle anzeigen</a>
|
||||
<h2 th:if="${patientPseudonym != null}">
|
||||
Betreffend Patienten-Pseudonym <span class="monospace" th:text="${patientPseudonym}">***</span>
|
||||
<a class="btn btn-blue" th:if="${patientPseudonym != null}" th:href="@{/}">Alle anzeigen</a>
|
||||
</h2>
|
||||
</div>
|
||||
|
||||
<div class="border">
|
||||
<div th:if="${patientId == null}" class="page-control">
|
||||
<div class="border" th:if="${requests.totalElements == 0}">
|
||||
<div class="notification info">Noch keine Anfragen eingegangen</div>
|
||||
</div>
|
||||
|
||||
<div class="border" th:if="${requests.totalElements > 0}">
|
||||
<div th:if="${patientPseudonym == null}" class="page-control">
|
||||
<a id="first-page-link" th:href="@{/(page=${0})}" title="Zum Anfang: Taste W" th:if="${not requests.isFirst()}">⇤</a><a th:if="${requests.isFirst()}">⇤</a>
|
||||
<a id="prev-page-link" th:href="@{/(page=${requests.getNumber() - 1})}" title="Seite zurück: Taste A" th:if="${not requests.isFirst()}">←</a><a th:if="${requests.isFirst()}">←</a>
|
||||
<span>Seite [[ ${requests.getNumber() + 1} ]] von [[ ${requests.getTotalPages()} ]]</span>
|
||||
<a id="next-page-link" th:href="@{/(page=${requests.getNumber() + 1})}" title="Seite vor: Taste D" th:if="${not requests.isLast()}">→</a><a th:if="${requests.isLast()}">→</a>
|
||||
<a id="last-page-link" th:href="@{/(page=${requests.getTotalPages() - 1})}" title="Zum Ende: Taste S" th:if="${not requests.isLast()}">⇥</a><a th:if="${requests.isLast()}">⇥</a>
|
||||
</div>
|
||||
<div th:if="${patientId != null}" class="page-control">
|
||||
<a id="first-page-link" th:href="@{/patient/{patientId}(patientId=${patientId},page=${0})}" title="Zum Anfang: Taste W" th:if="${not requests.isFirst()}">⇤</a><a th:if="${requests.isFirst()}">⇤</a>
|
||||
<a id="prev-page-link" th:href="@{/patient/{patientId}(patientId=${patientId},page=${requests.getNumber() - 1})}" title="Seite zurück: Taste A" th:if="${not requests.isFirst()}">←</a><a th:if="${requests.isFirst()}">←</a>
|
||||
<div th:if="${patientPseudonym != null}" class="page-control">
|
||||
<a id="first-page-link" th:href="@{/patient/{patientPseudonym}(patientPseudonym=${patientPseudonym},page=${0})}" title="Zum Anfang: Taste W" th:if="${not requests.isFirst()}">⇤</a><a th:if="${requests.isFirst()}">⇤</a>
|
||||
<a id="prev-page-link" th:href="@{/patient/{patientPseudonym}(patientPseudonym=${patientPseudonym},page=${requests.getNumber() - 1})}" title="Seite zurück: Taste A" th:if="${not requests.isFirst()}">←</a><a th:if="${requests.isFirst()}">←</a>
|
||||
<span>Seite [[ ${requests.getNumber() + 1} ]] von [[ ${requests.getTotalPages()} ]]</span>
|
||||
<a id="next-page-link" th:href="@{/patient/{patientId}(patientId=${patientId},page=${requests.getNumber() + 1})}" title="Seite vor: Taste D" th:if="${not requests.isLast()}">→</a><a th:if="${requests.isLast()}">→</a>
|
||||
<a id="last-page-link" th:href="@{/patient/{patientId}(patientId=${patientId},page=${requests.getTotalPages() - 1})}" title="Zum Ende: Taste S" th:if="${not requests.isLast()}">⇥</a><a th:if="${requests.isLast()}">⇥</a>
|
||||
<a id="next-page-link" th:href="@{/patient/{patientPseudonym}(patientPseudonym=${patientPseudonym},page=${requests.getNumber() + 1})}" title="Seite vor: Taste D" th:if="${not requests.isLast()}">→</a><a th:if="${requests.isLast()}">→</a>
|
||||
<a id="last-page-link" th:href="@{/patient/{patientPseudonym}(patientPseudonym=${patientPseudonym},page=${requests.getTotalPages() - 1})}" title="Zum Ende: Taste S" th:if="${not requests.isLast()}">⇥</a><a th:if="${requests.isLast()}">⇥</a>
|
||||
</div>
|
||||
<table class="paged">
|
||||
<thead>
|
||||
@ -57,11 +61,11 @@
|
||||
<th:block sec:authorize="not (hasRole('USER') or hasRole('ADMIN'))">[[ ${request.uuid} ]]</th:block>
|
||||
</td>
|
||||
<td><time th:datetime="${request.processedAt}">[[ ${request.processedAt} ]]</time></td>
|
||||
<td class="patient-id" th:if="${patientId != null}" sec:authorize="hasRole('USER') or hasRole('ADMIN')">
|
||||
[[ ${request.patientId} ]]
|
||||
<td class="patient-id" th:if="${patientPseudonym != null}" sec:authorize="hasRole('USER') or hasRole('ADMIN')">
|
||||
[[ ${request.patientPseudonym} ]]
|
||||
</td>
|
||||
<td class="patient-id" th:if="${patientId == null}" sec:authorize="hasRole('USER') or hasRole('ADMIN')">
|
||||
<a th:href="@{/patient/{pid}(pid=${request.patientId})}">[[ ${request.patientId} ]]</a>
|
||||
<td class="patient-id" th:if="${patientPseudonym == null}" sec:authorize="hasRole('USER') or hasRole('ADMIN')">
|
||||
<a th:href="@{/patient/{pid}(pid=${request.patientPseudonym})}">[[ ${request.patientPseudonym} ]]</a>
|
||||
</td>
|
||||
<td class="patient-id" sec:authorize="not (hasRole('USER') or hasRole('ADMIN'))">***</td>
|
||||
</tr>
|
||||
|
@ -31,7 +31,7 @@
|
||||
<td th:style="${request.type.value == 'delete'} ? 'color: red;'"><small>[[ ${request.type} ]]</small></td>
|
||||
<td>[[ ${request.uuid} ]]</td>
|
||||
<td><time th:datetime="${request.processedAt}">[[ ${request.processedAt} ]]</time></td>
|
||||
<td class="patient-id" sec:authorize="authenticated">[[ ${request.patientId} ]]</td>
|
||||
<td class="patient-id" sec:authorize="authenticated">[[ ${request.patientPseudonym} ]]</td>
|
||||
<td class="patient-id" sec:authorize="not authenticated">***</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
@ -47,7 +47,7 @@
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Schweregrad</th>
|
||||
<th>Beschreibung</th>
|
||||
<th>Beschreibung und Pfad</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
@ -56,7 +56,11 @@
|
||||
<td th:if="${issue.severity.value == 'warning'}" class="bg-yellow"><small>[[ ${issue.severity} ]]</small></td>
|
||||
<td th:if="${issue.severity.value == 'error'}" class="bg-red"><small>[[ ${issue.severity} ]]</small></td>
|
||||
<td th:if="${issue.severity.value == 'fatal'}" class="bg-red"><small>[[ ${issue.severity} ]]</small></td>
|
||||
<td>[[ ${issue.message} ]]</td>
|
||||
<td>
|
||||
<div class="issue-message">[[ ${issue.message} ]]</div>
|
||||
<div class="issue-path" th:if="${issue.path.isPresent()}">[[ ${issue.path.get()} ]]</div>
|
||||
<div class="issue-path" th:if="${issue.path.isEmpty()}"><i>Keine Angabe</i></div>
|
||||
</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
20
src/test/kotlin/dev/dnpm/etl/processor/helpers.kt
Normal file
20
src/test/kotlin/dev/dnpm/etl/processor/helpers.kt
Normal file
@ -0,0 +1,20 @@
|
||||
/*
|
||||
* This file is part of ETL-Processor
|
||||
*
|
||||
* Copyright (c) 2024 Comprehensive Cancer Center Mainfranken, Datenintegrationszentrum Philipps-Universität Marburg and Contributors
|
||||
*
|
||||
* This program is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
* by the Free Software Foundation, either version 3 of the License, or
|
||||
* (at your option) any later version.
|
||||
*
|
||||
* This program is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
* GNU Affero General Public License for more details.
|
||||
*
|
||||
* You should have received a copy of the GNU Affero General Public License
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
*/
|
||||
|
||||
package dev.dnpm.etl.processor
|
@ -1,7 +1,7 @@
|
||||
/*
|
||||
* This file is part of ETL-Processor
|
||||
*
|
||||
* Copyright (c) 2024 Comprehensive Cancer Center Mainfranken, Datenintegrationszentrum Philipps-Universität Marburg and Contributors
|
||||
* Copyright (c) 2025 Comprehensive Cancer Center Mainfranken, Datenintegrationszentrum Philipps-Universität Marburg and Contributors
|
||||
*
|
||||
* This program is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
@ -23,6 +23,7 @@ import com.fasterxml.jackson.databind.ObjectMapper
|
||||
import de.ukw.ccc.bwhc.dto.Consent
|
||||
import de.ukw.ccc.bwhc.dto.MtbFile
|
||||
import de.ukw.ccc.bwhc.dto.Patient
|
||||
import dev.dnpm.etl.processor.CustomMediaType
|
||||
import dev.dnpm.etl.processor.services.RequestProcessor
|
||||
import org.apache.kafka.clients.consumer.ConsumerRecord
|
||||
import org.apache.kafka.common.header.internals.RecordHeader
|
||||
@ -31,10 +32,10 @@ import org.apache.kafka.common.record.TimestampType
|
||||
import org.junit.jupiter.api.BeforeEach
|
||||
import org.junit.jupiter.api.Test
|
||||
import org.junit.jupiter.api.extension.ExtendWith
|
||||
import org.mockito.ArgumentMatchers.anyString
|
||||
import org.mockito.Mock
|
||||
import org.mockito.junit.jupiter.MockitoExtension
|
||||
import org.mockito.kotlin.any
|
||||
import org.mockito.kotlin.anyValueClass
|
||||
import org.mockito.kotlin.times
|
||||
import org.mockito.kotlin.verify
|
||||
import java.util.*
|
||||
@ -63,9 +64,17 @@ class KafkaInputListenerTest {
|
||||
.withConsent(Consent.builder().withStatus(Consent.Status.ACTIVE).build())
|
||||
.build()
|
||||
|
||||
kafkaInputListener.onMessage(ConsumerRecord("testtopic", 0, 0, "", this.objectMapper.writeValueAsString(mtbFile)))
|
||||
kafkaInputListener.onMessage(
|
||||
ConsumerRecord(
|
||||
"testtopic",
|
||||
0,
|
||||
0,
|
||||
"",
|
||||
this.objectMapper.writeValueAsString(mtbFile)
|
||||
)
|
||||
)
|
||||
|
||||
verify(requestProcessor, times(1)).processMtbFile(any())
|
||||
verify(requestProcessor, times(1)).processMtbFile(any<MtbFile>())
|
||||
}
|
||||
|
||||
@Test
|
||||
@ -75,9 +84,17 @@ class KafkaInputListenerTest {
|
||||
.withConsent(Consent.builder().withStatus(Consent.Status.REJECTED).build())
|
||||
.build()
|
||||
|
||||
kafkaInputListener.onMessage(ConsumerRecord("testtopic", 0, 0, "", this.objectMapper.writeValueAsString(mtbFile)))
|
||||
kafkaInputListener.onMessage(
|
||||
ConsumerRecord(
|
||||
"testtopic",
|
||||
0,
|
||||
0,
|
||||
"",
|
||||
this.objectMapper.writeValueAsString(mtbFile)
|
||||
)
|
||||
)
|
||||
|
||||
verify(requestProcessor, times(1)).processDeletion(anyString())
|
||||
verify(requestProcessor, times(1)).processDeletion(anyValueClass())
|
||||
}
|
||||
|
||||
@Test
|
||||
@ -89,10 +106,22 @@ class KafkaInputListenerTest {
|
||||
|
||||
val headers = RecordHeaders(listOf(RecordHeader("requestId", UUID.randomUUID().toString().toByteArray())))
|
||||
kafkaInputListener.onMessage(
|
||||
ConsumerRecord("testtopic", 0, 0, -1L, TimestampType.NO_TIMESTAMP_TYPE, -1, -1, "", this.objectMapper.writeValueAsString(mtbFile), headers, Optional.empty())
|
||||
ConsumerRecord(
|
||||
"testtopic",
|
||||
0,
|
||||
0,
|
||||
-1L,
|
||||
TimestampType.NO_TIMESTAMP_TYPE,
|
||||
-1,
|
||||
-1,
|
||||
"",
|
||||
this.objectMapper.writeValueAsString(mtbFile),
|
||||
headers,
|
||||
Optional.empty()
|
||||
)
|
||||
)
|
||||
|
||||
verify(requestProcessor, times(1)).processMtbFile(any(), anyString())
|
||||
verify(requestProcessor, times(1)).processMtbFile(any<MtbFile>(), anyValueClass())
|
||||
}
|
||||
|
||||
@Test
|
||||
@ -104,9 +133,52 @@ class KafkaInputListenerTest {
|
||||
|
||||
val headers = RecordHeaders(listOf(RecordHeader("requestId", UUID.randomUUID().toString().toByteArray())))
|
||||
kafkaInputListener.onMessage(
|
||||
ConsumerRecord("testtopic", 0, 0, -1L, TimestampType.NO_TIMESTAMP_TYPE, -1, -1, "", this.objectMapper.writeValueAsString(mtbFile), headers, Optional.empty())
|
||||
ConsumerRecord(
|
||||
"testtopic",
|
||||
0,
|
||||
0,
|
||||
-1L,
|
||||
TimestampType.NO_TIMESTAMP_TYPE,
|
||||
-1,
|
||||
-1,
|
||||
"",
|
||||
this.objectMapper.writeValueAsString(mtbFile),
|
||||
headers,
|
||||
Optional.empty()
|
||||
)
|
||||
verify(requestProcessor, times(1)).processDeletion(anyString(), anyString())
|
||||
)
|
||||
verify(requestProcessor, times(1)).processDeletion(anyValueClass(), anyValueClass())
|
||||
}
|
||||
|
||||
@Test
|
||||
fun shouldNotProcessDnpmV2Request() {
|
||||
val mtbFile = MtbFile.builder()
|
||||
.withPatient(Patient.builder().withId("DUMMY_12345678").build())
|
||||
.withConsent(Consent.builder().withStatus(Consent.Status.REJECTED).build())
|
||||
.build()
|
||||
|
||||
val headers = RecordHeaders(
|
||||
listOf(
|
||||
RecordHeader("requestId", UUID.randomUUID().toString().toByteArray()),
|
||||
RecordHeader("contentType", CustomMediaType.APPLICATION_VND_DNPM_V2_MTB_JSON_VALUE.toByteArray())
|
||||
)
|
||||
)
|
||||
kafkaInputListener.onMessage(
|
||||
ConsumerRecord(
|
||||
"testtopic",
|
||||
0,
|
||||
0,
|
||||
-1L,
|
||||
TimestampType.NO_TIMESTAMP_TYPE,
|
||||
-1,
|
||||
-1,
|
||||
"",
|
||||
this.objectMapper.writeValueAsString(mtbFile),
|
||||
headers,
|
||||
Optional.empty()
|
||||
)
|
||||
)
|
||||
verify(requestProcessor, times(0)).processDeletion(anyValueClass(), anyValueClass())
|
||||
}
|
||||
|
||||
}
|
@ -1,7 +1,7 @@
|
||||
/*
|
||||
* This file is part of ETL-Processor
|
||||
*
|
||||
* Copyright (c) 2024 Comprehensive Cancer Center Mainfranken, Datenintegrationszentrum Philipps-Universität Marburg and Contributors
|
||||
* Copyright (c) 2025 Comprehensive Cancer Center Mainfranken, Datenintegrationszentrum Philipps-Universität Marburg and Contributors
|
||||
*
|
||||
* This program is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
@ -21,9 +21,11 @@ package dev.dnpm.etl.processor.input
|
||||
|
||||
import com.fasterxml.jackson.databind.ObjectMapper
|
||||
import de.ukw.ccc.bwhc.dto.*
|
||||
import dev.dnpm.etl.processor.CustomMediaType
|
||||
import dev.dnpm.etl.processor.services.RequestProcessor
|
||||
import org.assertj.core.api.Assertions.assertThat
|
||||
import dev.pcvolkmer.mv64e.mtb.Mtb
|
||||
import org.junit.jupiter.api.BeforeEach
|
||||
import org.junit.jupiter.api.Nested
|
||||
import org.junit.jupiter.api.Test
|
||||
import org.junit.jupiter.api.extension.ExtendWith
|
||||
import org.mockito.Mock
|
||||
@ -31,7 +33,8 @@ import org.mockito.Mockito.times
|
||||
import org.mockito.Mockito.verify
|
||||
import org.mockito.junit.jupiter.MockitoExtension
|
||||
import org.mockito.kotlin.any
|
||||
import org.mockito.kotlin.argumentCaptor
|
||||
import org.mockito.kotlin.anyValueClass
|
||||
import org.springframework.core.io.ClassPathResource
|
||||
import org.springframework.http.MediaType
|
||||
import org.springframework.test.web.servlet.MockMvc
|
||||
import org.springframework.test.web.servlet.delete
|
||||
@ -41,12 +44,15 @@ import org.springframework.test.web.servlet.setup.MockMvcBuilders
|
||||
@ExtendWith(MockitoExtension::class)
|
||||
class MtbFileRestControllerTest {
|
||||
|
||||
private val objectMapper = ObjectMapper()
|
||||
|
||||
@Nested
|
||||
inner class BwhcRequests {
|
||||
|
||||
private lateinit var mockMvc: MockMvc
|
||||
|
||||
private lateinit var requestProcessor: RequestProcessor
|
||||
|
||||
private val objectMapper = ObjectMapper()
|
||||
|
||||
@BeforeEach
|
||||
fun setup(
|
||||
@Mock requestProcessor: RequestProcessor
|
||||
@ -57,33 +63,9 @@ class MtbFileRestControllerTest {
|
||||
}
|
||||
|
||||
@Test
|
||||
fun shouldProcessMtbFilePostRequest() {
|
||||
val mtbFile = MtbFile.builder()
|
||||
.withPatient(
|
||||
Patient.builder()
|
||||
.withId("TEST_12345678")
|
||||
.withBirthDate("2000-08-08")
|
||||
.withGender(Patient.Gender.MALE)
|
||||
.build()
|
||||
)
|
||||
.withConsent(
|
||||
Consent.builder()
|
||||
.withId("1")
|
||||
.withStatus(Consent.Status.ACTIVE)
|
||||
.withPatient("TEST_12345678")
|
||||
.build()
|
||||
)
|
||||
.withEpisode(
|
||||
Episode.builder()
|
||||
.withId("1")
|
||||
.withPatient("TEST_12345678")
|
||||
.withPeriod(PeriodStart("2023-08-08"))
|
||||
.build()
|
||||
)
|
||||
.build()
|
||||
|
||||
fun shouldProcessPostRequest() {
|
||||
mockMvc.post("/mtbfile") {
|
||||
content = objectMapper.writeValueAsString(mtbFile)
|
||||
content = objectMapper.writeValueAsString(bwhcMtbFileContent(Consent.Status.ACTIVE))
|
||||
contentType = MediaType.APPLICATION_JSON
|
||||
}.andExpect {
|
||||
status {
|
||||
@ -91,37 +73,13 @@ class MtbFileRestControllerTest {
|
||||
}
|
||||
}
|
||||
|
||||
verify(requestProcessor, times(1)).processMtbFile(any())
|
||||
verify(requestProcessor, times(1)).processMtbFile(any<MtbFile>())
|
||||
}
|
||||
|
||||
@Test
|
||||
fun shouldProcessMtbFilePostRequestWithRejectedConsent() {
|
||||
val mtbFile = MtbFile.builder()
|
||||
.withPatient(
|
||||
Patient.builder()
|
||||
.withId("TEST_12345678")
|
||||
.withBirthDate("2000-08-08")
|
||||
.withGender(Patient.Gender.MALE)
|
||||
.build()
|
||||
)
|
||||
.withConsent(
|
||||
Consent.builder()
|
||||
.withId("1")
|
||||
.withStatus(Consent.Status.REJECTED)
|
||||
.withPatient("TEST_12345678")
|
||||
.build()
|
||||
)
|
||||
.withEpisode(
|
||||
Episode.builder()
|
||||
.withId("1")
|
||||
.withPatient("TEST_12345678")
|
||||
.withPeriod(PeriodStart("2023-08-08"))
|
||||
.build()
|
||||
)
|
||||
.build()
|
||||
|
||||
fun shouldProcessPostRequestWithRejectedConsent() {
|
||||
mockMvc.post("/mtbfile") {
|
||||
content = objectMapper.writeValueAsString(mtbFile)
|
||||
content = objectMapper.writeValueAsString(bwhcMtbFileContent(Consent.Status.REJECTED))
|
||||
contentType = MediaType.APPLICATION_JSON
|
||||
}.andExpect {
|
||||
status {
|
||||
@ -129,22 +87,134 @@ class MtbFileRestControllerTest {
|
||||
}
|
||||
}
|
||||
|
||||
val captor = argumentCaptor<String>()
|
||||
verify(requestProcessor, times(1)).processDeletion(captor.capture())
|
||||
assertThat(captor.firstValue).isEqualTo("TEST_12345678")
|
||||
verify(requestProcessor, times(1)).processDeletion(anyValueClass())
|
||||
}
|
||||
|
||||
@Test
|
||||
fun shouldProcessMtbFileDeleteRequest() {
|
||||
fun shouldProcessDeleteRequest() {
|
||||
mockMvc.delete("/mtbfile/TEST_12345678").andExpect {
|
||||
status {
|
||||
isAccepted()
|
||||
}
|
||||
}
|
||||
|
||||
val captor = argumentCaptor<String>()
|
||||
verify(requestProcessor, times(1)).processDeletion(captor.capture())
|
||||
assertThat(captor.firstValue).isEqualTo("TEST_12345678")
|
||||
verify(requestProcessor, times(1)).processDeletion(anyValueClass())
|
||||
}
|
||||
}
|
||||
|
||||
@Nested
|
||||
inner class BwhcRequestsWithAlias {
|
||||
|
||||
private lateinit var mockMvc: MockMvc
|
||||
|
||||
private lateinit var requestProcessor: RequestProcessor
|
||||
|
||||
@BeforeEach
|
||||
fun setup(
|
||||
@Mock requestProcessor: RequestProcessor
|
||||
) {
|
||||
this.requestProcessor = requestProcessor
|
||||
val controller = MtbFileRestController(requestProcessor)
|
||||
this.mockMvc = MockMvcBuilders.standaloneSetup(controller).build()
|
||||
}
|
||||
|
||||
@Test
|
||||
fun shouldProcessPostRequest() {
|
||||
mockMvc.post("/mtb") {
|
||||
content = objectMapper.writeValueAsString(bwhcMtbFileContent(Consent.Status.ACTIVE))
|
||||
contentType = MediaType.APPLICATION_JSON
|
||||
}.andExpect {
|
||||
status {
|
||||
isAccepted()
|
||||
}
|
||||
}
|
||||
|
||||
verify(requestProcessor, times(1)).processMtbFile(any<MtbFile>())
|
||||
}
|
||||
|
||||
@Test
|
||||
fun shouldProcessPostRequestWithRejectedConsent() {
|
||||
mockMvc.post("/mtb") {
|
||||
content = objectMapper.writeValueAsString(bwhcMtbFileContent(Consent.Status.REJECTED))
|
||||
contentType = MediaType.APPLICATION_JSON
|
||||
}.andExpect {
|
||||
status {
|
||||
isAccepted()
|
||||
}
|
||||
}
|
||||
|
||||
verify(requestProcessor, times(1)).processDeletion(anyValueClass())
|
||||
}
|
||||
|
||||
@Test
|
||||
fun shouldProcessDeleteRequest() {
|
||||
mockMvc.delete("/mtb/TEST_12345678").andExpect {
|
||||
status {
|
||||
isAccepted()
|
||||
}
|
||||
}
|
||||
|
||||
verify(requestProcessor, times(1)).processDeletion(anyValueClass())
|
||||
}
|
||||
}
|
||||
|
||||
@Nested
|
||||
inner class RequestsForDnpmDataModel21 {
|
||||
|
||||
private lateinit var mockMvc: MockMvc
|
||||
|
||||
private lateinit var requestProcessor: RequestProcessor
|
||||
|
||||
@BeforeEach
|
||||
fun setup(
|
||||
@Mock requestProcessor: RequestProcessor
|
||||
) {
|
||||
this.requestProcessor = requestProcessor
|
||||
val controller = MtbFileRestController(requestProcessor)
|
||||
this.mockMvc = MockMvcBuilders.standaloneSetup(controller).build()
|
||||
}
|
||||
|
||||
@Test
|
||||
fun shouldRespondPostRequest() {
|
||||
val mtbFileContent = ClassPathResource("mv64e-mtb-fake-patient.json").inputStream.readAllBytes().toString(Charsets.UTF_8)
|
||||
|
||||
mockMvc.post("/mtb") {
|
||||
content = mtbFileContent
|
||||
contentType = CustomMediaType.APPLICATION_VND_DNPM_V2_MTB_JSON
|
||||
}.andExpect {
|
||||
status {
|
||||
isAccepted()
|
||||
}
|
||||
}
|
||||
|
||||
verify(requestProcessor, times(1)).processMtbFile(any<Mtb>())
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
companion object {
|
||||
fun bwhcMtbFileContent(consentStatus: Consent.Status) = MtbFile.builder()
|
||||
.withPatient(
|
||||
Patient.builder()
|
||||
.withId("TEST_12345678")
|
||||
.withBirthDate("2000-08-08")
|
||||
.withGender(Patient.Gender.MALE)
|
||||
.build()
|
||||
)
|
||||
.withConsent(
|
||||
Consent.builder()
|
||||
.withId("1")
|
||||
.withStatus(consentStatus)
|
||||
.withPatient("TEST_12345678")
|
||||
.build()
|
||||
)
|
||||
.withEpisode(
|
||||
Episode.builder()
|
||||
.withId("1")
|
||||
.withPatient("TEST_12345678")
|
||||
.withPeriod(PeriodStart("2023-08-08"))
|
||||
.build()
|
||||
)
|
||||
.build()
|
||||
}
|
||||
}
|
||||
|
@ -1,7 +1,7 @@
|
||||
/*
|
||||
* This file is part of ETL-Processor
|
||||
*
|
||||
* Copyright (c) 2024 Comprehensive Cancer Center Mainfranken, Datenintegrationszentrum Philipps-Universität Marburg and Contributors
|
||||
* Copyright (c) 2025 Comprehensive Cancer Center Mainfranken, Datenintegrationszentrum Philipps-Universität Marburg and Contributors
|
||||
*
|
||||
* This program is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
@ -21,18 +21,25 @@ package dev.dnpm.etl.processor.output
|
||||
|
||||
import com.fasterxml.jackson.databind.ObjectMapper
|
||||
import de.ukw.ccc.bwhc.dto.*
|
||||
import de.ukw.ccc.bwhc.dto.Patient
|
||||
import dev.dnpm.etl.processor.CustomMediaType
|
||||
import dev.dnpm.etl.processor.PatientPseudonym
|
||||
import dev.dnpm.etl.processor.RequestId
|
||||
import dev.dnpm.etl.processor.config.KafkaProperties
|
||||
import dev.dnpm.etl.processor.monitoring.RequestStatus
|
||||
import dev.pcvolkmer.mv64e.mtb.*
|
||||
import org.apache.kafka.clients.producer.ProducerRecord
|
||||
import org.assertj.core.api.Assertions.assertThat
|
||||
import org.junit.jupiter.api.BeforeEach
|
||||
import org.junit.jupiter.api.Nested
|
||||
import org.junit.jupiter.api.Test
|
||||
import org.junit.jupiter.api.extension.ExtendWith
|
||||
import org.junit.jupiter.params.ParameterizedTest
|
||||
import org.junit.jupiter.params.provider.MethodSource
|
||||
import org.mockito.ArgumentMatchers.anyString
|
||||
import org.mockito.Mock
|
||||
import org.mockito.junit.jupiter.MockitoExtension
|
||||
import org.mockito.kotlin.*
|
||||
import org.springframework.http.MediaType
|
||||
import org.springframework.kafka.core.KafkaTemplate
|
||||
import org.springframework.kafka.support.SendResult
|
||||
import org.springframework.retry.policy.SimpleRetryPolicy
|
||||
@ -43,6 +50,9 @@ import java.util.concurrent.ExecutionException
|
||||
@ExtendWith(MockitoExtension::class)
|
||||
class KafkaMtbFileSenderTest {
|
||||
|
||||
@Nested
|
||||
inner class BwhcV1Record {
|
||||
|
||||
private lateinit var kafkaTemplate: KafkaTemplate<String, String>
|
||||
|
||||
private lateinit var kafkaMtbFileSender: KafkaMtbFileSender
|
||||
@ -63,67 +73,69 @@ class KafkaMtbFileSenderTest {
|
||||
}
|
||||
|
||||
@ParameterizedTest
|
||||
@MethodSource("requestWithResponseSource")
|
||||
@MethodSource("dev.dnpm.etl.processor.output.KafkaMtbFileSenderTest#requestWithResponseSource")
|
||||
fun shouldSendMtbFileRequestAndReturnExpectedState(testData: TestData) {
|
||||
doAnswer {
|
||||
if (null != testData.exception) {
|
||||
throw testData.exception
|
||||
}
|
||||
completedFuture(SendResult<String, String>(null, null))
|
||||
}.whenever(kafkaTemplate).send(anyString(), anyString(), anyString())
|
||||
}.whenever(kafkaTemplate).send(any<ProducerRecord<String, String>>())
|
||||
|
||||
val response = kafkaMtbFileSender.send(MtbFileSender.MtbFileRequest("TestID", mtbFile(Consent.Status.ACTIVE)))
|
||||
val response = kafkaMtbFileSender.send(BwhcV1MtbFileRequest(TEST_REQUEST_ID, bwhcV1MtbFile(Consent.Status.ACTIVE)))
|
||||
assertThat(response.status).isEqualTo(testData.requestStatus)
|
||||
}
|
||||
|
||||
@ParameterizedTest
|
||||
@MethodSource("requestWithResponseSource")
|
||||
@MethodSource("dev.dnpm.etl.processor.output.KafkaMtbFileSenderTest#requestWithResponseSource")
|
||||
fun shouldSendDeleteRequestAndReturnExpectedState(testData: TestData) {
|
||||
doAnswer {
|
||||
if (null != testData.exception) {
|
||||
throw testData.exception
|
||||
}
|
||||
completedFuture(SendResult<String, String>(null, null))
|
||||
}.whenever(kafkaTemplate).send(anyString(), anyString(), anyString())
|
||||
}.whenever(kafkaTemplate).send(any<ProducerRecord<String, String>>())
|
||||
|
||||
val response = kafkaMtbFileSender.send(MtbFileSender.DeleteRequest("TestID", "PID"))
|
||||
val response = kafkaMtbFileSender.send(DeleteRequest(TEST_REQUEST_ID, TEST_PATIENT_PSEUDONYM))
|
||||
assertThat(response.status).isEqualTo(testData.requestStatus)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun shouldSendMtbFileRequestWithCorrectKeyAndBody() {
|
||||
fun shouldSendMtbFileRequestWithCorrectKeyAndHeaderAndBody() {
|
||||
doAnswer {
|
||||
completedFuture(SendResult<String, String>(null, null))
|
||||
}.whenever(kafkaTemplate).send(anyString(), anyString(), anyString())
|
||||
}.whenever(kafkaTemplate).send(any<ProducerRecord<String, String>>())
|
||||
|
||||
kafkaMtbFileSender.send(MtbFileSender.MtbFileRequest("TestID", mtbFile(Consent.Status.ACTIVE)))
|
||||
kafkaMtbFileSender.send(BwhcV1MtbFileRequest(TEST_REQUEST_ID, bwhcV1MtbFile(Consent.Status.ACTIVE)))
|
||||
|
||||
val captor = argumentCaptor<String>()
|
||||
verify(kafkaTemplate, times(1)).send(anyString(), captor.capture(), captor.capture())
|
||||
assertThat(captor.firstValue).isNotNull
|
||||
assertThat(captor.firstValue).isEqualTo("{\"pid\": \"PID\"}")
|
||||
assertThat(captor.secondValue).isNotNull
|
||||
assertThat(captor.secondValue).isEqualTo(objectMapper.writeValueAsString(kafkaRecordData("TestID", Consent.Status.ACTIVE)))
|
||||
val captor = argumentCaptor<ProducerRecord<String, String>>()
|
||||
verify(kafkaTemplate, times(1)).send(captor.capture())
|
||||
assertThat(captor.firstValue.key()).isNotNull
|
||||
assertThat(captor.firstValue.key()).isEqualTo("{\"pid\": \"PID\"}")
|
||||
assertThat(captor.firstValue.headers().headers("contentType")).isNotNull
|
||||
assertThat(captor.firstValue.headers().headers("contentType")?.firstOrNull()?.value()).isEqualTo(MediaType.APPLICATION_JSON_VALUE.toByteArray())
|
||||
assertThat(captor.firstValue.value()).isNotNull
|
||||
assertThat(captor.firstValue.value()).isEqualTo(objectMapper.writeValueAsString(bwhcV1kafkaRecordData(TEST_REQUEST_ID, Consent.Status.ACTIVE)))
|
||||
}
|
||||
|
||||
@Test
|
||||
fun shouldSendDeleteRequestWithCorrectKeyAndBody() {
|
||||
doAnswer {
|
||||
completedFuture(SendResult<String, String>(null, null))
|
||||
}.whenever(kafkaTemplate).send(anyString(), anyString(), anyString())
|
||||
}.whenever(kafkaTemplate).send(any<ProducerRecord<String, String>>())
|
||||
|
||||
kafkaMtbFileSender.send(MtbFileSender.DeleteRequest("TestID", "PID"))
|
||||
kafkaMtbFileSender.send(DeleteRequest(TEST_REQUEST_ID, TEST_PATIENT_PSEUDONYM))
|
||||
|
||||
val captor = argumentCaptor<String>()
|
||||
verify(kafkaTemplate, times(1)).send(anyString(), captor.capture(), captor.capture())
|
||||
assertThat(captor.firstValue).isNotNull
|
||||
assertThat(captor.firstValue).isEqualTo("{\"pid\": \"PID\"}")
|
||||
assertThat(captor.secondValue).isNotNull
|
||||
assertThat(captor.secondValue).isEqualTo(objectMapper.writeValueAsString(kafkaRecordData("TestID", Consent.Status.REJECTED)))
|
||||
val captor = argumentCaptor<ProducerRecord<String, String>>()
|
||||
verify(kafkaTemplate, times(1)).send(captor.capture())
|
||||
assertThat(captor.firstValue.key()).isNotNull
|
||||
assertThat(captor.firstValue.key()).isEqualTo("{\"pid\": \"PID\"}")
|
||||
assertThat(captor.firstValue.value()).isNotNull
|
||||
assertThat(captor.firstValue.value()).isEqualTo(objectMapper.writeValueAsString(bwhcV1kafkaRecordData(TEST_REQUEST_ID, Consent.Status.REJECTED)))
|
||||
}
|
||||
|
||||
@ParameterizedTest
|
||||
@MethodSource("requestWithResponseSource")
|
||||
@MethodSource("dev.dnpm.etl.processor.output.KafkaMtbFileSenderTest#requestWithResponseSource")
|
||||
fun shouldRetryOnMtbFileKafkaSendError(testData: TestData) {
|
||||
val kafkaProperties = KafkaProperties("testtopic")
|
||||
val retryTemplate = RetryTemplateBuilder().customPolicy(SimpleRetryPolicy(3)).build()
|
||||
@ -134,9 +146,9 @@ class KafkaMtbFileSenderTest {
|
||||
throw testData.exception
|
||||
}
|
||||
completedFuture(SendResult<String, String>(null, null))
|
||||
}.whenever(kafkaTemplate).send(anyString(), anyString(), anyString())
|
||||
}.whenever(kafkaTemplate).send(any<ProducerRecord<String, String>>())
|
||||
|
||||
kafkaMtbFileSender.send(MtbFileSender.MtbFileRequest("TestID", mtbFile(Consent.Status.ACTIVE)))
|
||||
kafkaMtbFileSender.send(BwhcV1MtbFileRequest(TEST_REQUEST_ID, bwhcV1MtbFile(Consent.Status.ACTIVE)))
|
||||
|
||||
val expectedCount = when (testData.exception) {
|
||||
// OK - No Retry
|
||||
@ -145,11 +157,11 @@ class KafkaMtbFileSenderTest {
|
||||
else -> times(3)
|
||||
}
|
||||
|
||||
verify(kafkaTemplate, expectedCount).send(anyString(), anyString(), anyString())
|
||||
verify(kafkaTemplate, expectedCount).send(any<ProducerRecord<String, String>>())
|
||||
}
|
||||
|
||||
@ParameterizedTest
|
||||
@MethodSource("requestWithResponseSource")
|
||||
@MethodSource("dev.dnpm.etl.processor.output.KafkaMtbFileSenderTest#requestWithResponseSource")
|
||||
fun shouldRetryOnDeleteKafkaSendError(testData: TestData) {
|
||||
val kafkaProperties = KafkaProperties("testtopic")
|
||||
val retryTemplate = RetryTemplateBuilder().customPolicy(SimpleRetryPolicy(3)).build()
|
||||
@ -160,9 +172,9 @@ class KafkaMtbFileSenderTest {
|
||||
throw testData.exception
|
||||
}
|
||||
completedFuture(SendResult<String, String>(null, null))
|
||||
}.whenever(kafkaTemplate).send(anyString(), anyString(), anyString())
|
||||
}.whenever(kafkaTemplate).send(any<ProducerRecord<String, String>>())
|
||||
|
||||
kafkaMtbFileSender.send(MtbFileSender.DeleteRequest("TestID", "PID"))
|
||||
kafkaMtbFileSender.send(DeleteRequest(TEST_REQUEST_ID, TEST_PATIENT_PSEUDONYM))
|
||||
|
||||
val expectedCount = when (testData.exception) {
|
||||
// OK - No Retry
|
||||
@ -171,11 +183,98 @@ class KafkaMtbFileSenderTest {
|
||||
else -> times(3)
|
||||
}
|
||||
|
||||
verify(kafkaTemplate, expectedCount).send(anyString(), anyString(), anyString())
|
||||
verify(kafkaTemplate, expectedCount).send(any<ProducerRecord<String, String>>())
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
@Nested
|
||||
inner class DnpmV2Record {
|
||||
|
||||
private lateinit var kafkaTemplate: KafkaTemplate<String, String>
|
||||
|
||||
private lateinit var kafkaMtbFileSender: KafkaMtbFileSender
|
||||
|
||||
private lateinit var objectMapper: ObjectMapper
|
||||
|
||||
@BeforeEach
|
||||
fun setup(
|
||||
@Mock kafkaTemplate: KafkaTemplate<String, String>
|
||||
) {
|
||||
val kafkaProperties = KafkaProperties("testtopic")
|
||||
val retryTemplate = RetryTemplateBuilder().customPolicy(SimpleRetryPolicy(1)).build()
|
||||
|
||||
this.objectMapper = ObjectMapper()
|
||||
this.kafkaTemplate = kafkaTemplate
|
||||
|
||||
this.kafkaMtbFileSender = KafkaMtbFileSender(kafkaTemplate, kafkaProperties, retryTemplate, objectMapper)
|
||||
}
|
||||
|
||||
@ParameterizedTest
|
||||
@MethodSource("dev.dnpm.etl.processor.output.KafkaMtbFileSenderTest#requestWithResponseSource")
|
||||
fun shouldSendMtbFileRequestAndReturnExpectedState(testData: TestData) {
|
||||
doAnswer {
|
||||
if (null != testData.exception) {
|
||||
throw testData.exception
|
||||
}
|
||||
completedFuture(SendResult<String, String>(null, null))
|
||||
}.whenever(kafkaTemplate).send(any<ProducerRecord<String, String>>())
|
||||
|
||||
val response = kafkaMtbFileSender.send(DnpmV2MtbFileRequest(TEST_REQUEST_ID, dnpmV2MtbFile()))
|
||||
assertThat(response.status).isEqualTo(testData.requestStatus)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun shouldSendMtbFileRequestWithCorrectKeyAndHeaderAndBody() {
|
||||
doAnswer {
|
||||
completedFuture(SendResult<String, String>(null, null))
|
||||
}.whenever(kafkaTemplate).send(any<ProducerRecord<String, String>>())
|
||||
|
||||
kafkaMtbFileSender.send(DnpmV2MtbFileRequest(TEST_REQUEST_ID, dnpmV2MtbFile()))
|
||||
|
||||
val captor = argumentCaptor<ProducerRecord<String, String>>()
|
||||
verify(kafkaTemplate, times(1)).send(captor.capture())
|
||||
assertThat(captor.firstValue.key()).isNotNull
|
||||
assertThat(captor.firstValue.key()).isEqualTo("{\"pid\": \"PID\"}")
|
||||
assertThat(captor.firstValue.headers().headers("contentType")).isNotNull
|
||||
assertThat(captor.firstValue.headers().headers("contentType")?.firstOrNull()?.value()).isEqualTo(CustomMediaType.APPLICATION_VND_DNPM_V2_MTB_JSON_VALUE.toByteArray())
|
||||
assertThat(captor.firstValue.value()).isNotNull
|
||||
assertThat(captor.firstValue.value()).isEqualTo(objectMapper.writeValueAsString(dnmpV2kafkaRecordData(TEST_REQUEST_ID)))
|
||||
}
|
||||
|
||||
@ParameterizedTest
|
||||
@MethodSource("dev.dnpm.etl.processor.output.KafkaMtbFileSenderTest#requestWithResponseSource")
|
||||
fun shouldRetryOnMtbFileKafkaSendError(testData: TestData) {
|
||||
val kafkaProperties = KafkaProperties("testtopic")
|
||||
val retryTemplate = RetryTemplateBuilder().customPolicy(SimpleRetryPolicy(3)).build()
|
||||
this.kafkaMtbFileSender = KafkaMtbFileSender(this.kafkaTemplate, kafkaProperties, retryTemplate, this.objectMapper)
|
||||
|
||||
doAnswer {
|
||||
if (null != testData.exception) {
|
||||
throw testData.exception
|
||||
}
|
||||
completedFuture(SendResult<String, String>(null, null))
|
||||
}.whenever(kafkaTemplate).send(any<ProducerRecord<String, String>>())
|
||||
|
||||
kafkaMtbFileSender.send(DnpmV2MtbFileRequest(TEST_REQUEST_ID, dnpmV2MtbFile()))
|
||||
|
||||
val expectedCount = when (testData.exception) {
|
||||
// OK - No Retry
|
||||
null -> times(1)
|
||||
// Request failed - Retry max 3 times
|
||||
else -> times(3)
|
||||
}
|
||||
|
||||
verify(kafkaTemplate, expectedCount).send(any<ProducerRecord<String, String>>())
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
companion object {
|
||||
fun mtbFile(consentStatus: Consent.Status): MtbFile {
|
||||
val TEST_REQUEST_ID = RequestId("TestId")
|
||||
val TEST_PATIENT_PSEUDONYM = PatientPseudonym("PID")
|
||||
|
||||
fun bwhcV1MtbFile(consentStatus: Consent.Status): MtbFile {
|
||||
return if (consentStatus == Consent.Status.ACTIVE) {
|
||||
MtbFile.builder()
|
||||
.withPatient(
|
||||
@ -210,8 +309,31 @@ class KafkaMtbFileSenderTest {
|
||||
}.build()
|
||||
}
|
||||
|
||||
fun kafkaRecordData(requestId: String, consentStatus: Consent.Status): KafkaMtbFileSender.Data {
|
||||
return KafkaMtbFileSender.Data(requestId, mtbFile(consentStatus))
|
||||
fun dnpmV2MtbFile(): Mtb = Mtb.builder()
|
||||
.withPatient(
|
||||
dev.pcvolkmer.mv64e.mtb.Patient.builder()
|
||||
.withId("PID")
|
||||
.withBirthDate("2000-08-08")
|
||||
.withGender(CodingGender.builder().withCode(CodingGender.Code.MALE).build())
|
||||
.build()
|
||||
)
|
||||
.withEpisodesOfCare(
|
||||
listOf(
|
||||
MTBEpisodeOfCare.builder()
|
||||
.withId("1")
|
||||
.withPatient(Reference("PID"))
|
||||
.withPeriod(PeriodDate.builder().withStart("2023-08-08").build())
|
||||
.build()
|
||||
)
|
||||
)
|
||||
.build()
|
||||
|
||||
fun bwhcV1kafkaRecordData(requestId: RequestId, consentStatus: Consent.Status): MtbRequest {
|
||||
return BwhcV1MtbFileRequest(requestId, bwhcV1MtbFile(consentStatus))
|
||||
}
|
||||
|
||||
fun dnmpV2kafkaRecordData(requestId: RequestId): MtbRequest {
|
||||
return DnpmV2MtbFileRequest(requestId, dnpmV2MtbFile())
|
||||
}
|
||||
|
||||
data class TestData(val requestStatus: RequestStatus, val exception: Throwable? = null)
|
||||
|
@ -1,7 +1,7 @@
|
||||
/*
|
||||
* This file is part of ETL-Processor
|
||||
*
|
||||
* Copyright (c) 2024 Comprehensive Cancer Center Mainfranken, Datenintegrationszentrum Philipps-Universität Marburg and Contributors
|
||||
* Copyright (c) 2025 Comprehensive Cancer Center Mainfranken, Datenintegrationszentrum Philipps-Universität Marburg and Contributors
|
||||
*
|
||||
* This program is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
@ -19,52 +19,61 @@
|
||||
|
||||
package dev.dnpm.etl.processor.output
|
||||
|
||||
import com.fasterxml.jackson.databind.ObjectMapper
|
||||
import com.fasterxml.jackson.module.kotlin.KotlinModule
|
||||
import de.ukw.ccc.bwhc.dto.*
|
||||
import dev.dnpm.etl.processor.PatientPseudonym
|
||||
import dev.dnpm.etl.processor.RequestId
|
||||
import dev.dnpm.etl.processor.config.RestTargetProperties
|
||||
import dev.dnpm.etl.processor.monitoring.ReportService
|
||||
import dev.dnpm.etl.processor.monitoring.RequestStatus
|
||||
import org.assertj.core.api.Assertions.assertThat
|
||||
import org.junit.jupiter.api.BeforeEach
|
||||
import org.junit.jupiter.params.ParameterizedTest
|
||||
import org.junit.jupiter.params.provider.MethodSource
|
||||
import org.springframework.http.HttpHeaders
|
||||
import org.springframework.http.HttpMethod
|
||||
import org.springframework.http.HttpStatus
|
||||
import org.springframework.http.MediaType
|
||||
import org.springframework.retry.policy.SimpleRetryPolicy
|
||||
import org.springframework.retry.support.RetryTemplateBuilder
|
||||
import org.springframework.test.web.client.ExpectedCount
|
||||
import org.springframework.test.web.client.MockRestServiceServer
|
||||
import org.springframework.test.web.client.match.MockRestRequestMatchers.method
|
||||
import org.springframework.test.web.client.match.MockRestRequestMatchers.requestTo
|
||||
import org.springframework.test.web.client.match.MockRestRequestMatchers.*
|
||||
import org.springframework.test.web.client.response.MockRestResponseCreators.withStatus
|
||||
import org.springframework.web.client.RestTemplate
|
||||
|
||||
class RestMtbFileSenderTest {
|
||||
class RestBwhcMtbFileSenderTest {
|
||||
|
||||
private lateinit var mockRestServiceServer: MockRestServiceServer
|
||||
|
||||
private lateinit var restMtbFileSender: RestMtbFileSender
|
||||
|
||||
private var reportService = ReportService(ObjectMapper().registerModule(KotlinModule.Builder().build()))
|
||||
|
||||
@BeforeEach
|
||||
fun setup() {
|
||||
val restTemplate = RestTemplate()
|
||||
val restTargetProperties = RestTargetProperties("http://localhost:9000/mtbfile")
|
||||
val restTargetProperties = RestTargetProperties("http://localhost:9000/mtbfile", null, null)
|
||||
val retryTemplate = RetryTemplateBuilder().customPolicy(SimpleRetryPolicy(1)).build()
|
||||
|
||||
this.mockRestServiceServer = MockRestServiceServer.createServer(restTemplate)
|
||||
|
||||
this.restMtbFileSender = RestMtbFileSender(restTemplate, restTargetProperties, retryTemplate)
|
||||
this.restMtbFileSender =
|
||||
RestBwhcMtbFileSender(restTemplate, restTargetProperties, retryTemplate, reportService)
|
||||
}
|
||||
|
||||
@ParameterizedTest
|
||||
@MethodSource("deleteRequestWithResponseSource")
|
||||
fun shouldReturnExpectedResponseForDelete(requestWithResponse: RequestWithResponse) {
|
||||
this.mockRestServiceServer.expect {
|
||||
method(HttpMethod.DELETE)
|
||||
requestTo("/mtbfile")
|
||||
}.andRespond {
|
||||
this.mockRestServiceServer
|
||||
.expect(method(HttpMethod.DELETE))
|
||||
.andExpect(requestTo("http://localhost:9000/mtbfile/Patient/${TEST_PATIENT_PSEUDONYM.value}"))
|
||||
.andRespond {
|
||||
withStatus(requestWithResponse.httpStatus).body(requestWithResponse.body).createResponse(it)
|
||||
}
|
||||
|
||||
val response = restMtbFileSender.send(MtbFileSender.DeleteRequest("TestID", "PID"))
|
||||
val response = restMtbFileSender.send(DeleteRequest(TEST_REQUEST_ID, TEST_PATIENT_PSEUDONYM))
|
||||
assertThat(response.status).isEqualTo(requestWithResponse.response.status)
|
||||
assertThat(response.body).isEqualTo(requestWithResponse.response.body)
|
||||
}
|
||||
@ -72,14 +81,15 @@ class RestMtbFileSenderTest {
|
||||
@ParameterizedTest
|
||||
@MethodSource("mtbFileRequestWithResponseSource")
|
||||
fun shouldReturnExpectedResponseForMtbFilePost(requestWithResponse: RequestWithResponse) {
|
||||
this.mockRestServiceServer.expect {
|
||||
method(HttpMethod.POST)
|
||||
requestTo("/mtbfile")
|
||||
}.andRespond {
|
||||
this.mockRestServiceServer
|
||||
.expect(method(HttpMethod.POST))
|
||||
.andExpect(requestTo("http://localhost:9000/mtbfile/MTBFile"))
|
||||
.andExpect(header(HttpHeaders.CONTENT_TYPE, MediaType.APPLICATION_JSON_VALUE))
|
||||
.andRespond {
|
||||
withStatus(requestWithResponse.httpStatus).body(requestWithResponse.body).createResponse(it)
|
||||
}
|
||||
|
||||
val response = restMtbFileSender.send(MtbFileSender.MtbFileRequest("TestID", mtbFile))
|
||||
val response = restMtbFileSender.send(BwhcV1MtbFileRequest(TEST_REQUEST_ID, mtbFile))
|
||||
assertThat(response.status).isEqualTo(requestWithResponse.response.status)
|
||||
assertThat(response.body).isEqualTo(requestWithResponse.response.body)
|
||||
}
|
||||
@ -88,11 +98,12 @@ class RestMtbFileSenderTest {
|
||||
@MethodSource("mtbFileRequestWithResponseSource")
|
||||
fun shouldRetryOnMtbFileHttpRequestError(requestWithResponse: RequestWithResponse) {
|
||||
val restTemplate = RestTemplate()
|
||||
val restTargetProperties = RestTargetProperties("http://localhost:9000/mtbfile")
|
||||
val restTargetProperties = RestTargetProperties("http://localhost:9000/mtbfile", null, null)
|
||||
val retryTemplate = RetryTemplateBuilder().customPolicy(SimpleRetryPolicy(3)).build()
|
||||
|
||||
this.mockRestServiceServer = MockRestServiceServer.createServer(restTemplate)
|
||||
this.restMtbFileSender = RestMtbFileSender(restTemplate, restTargetProperties, retryTemplate)
|
||||
this.restMtbFileSender =
|
||||
RestBwhcMtbFileSender(restTemplate, restTargetProperties, retryTemplate, reportService)
|
||||
|
||||
val expectedCount = when (requestWithResponse.httpStatus) {
|
||||
// OK - No Retry
|
||||
@ -101,14 +112,14 @@ class RestMtbFileSenderTest {
|
||||
else -> ExpectedCount.max(3)
|
||||
}
|
||||
|
||||
this.mockRestServiceServer.expect(expectedCount) {
|
||||
method(HttpMethod.POST)
|
||||
requestTo("/mtbfile")
|
||||
}.andRespond {
|
||||
this.mockRestServiceServer
|
||||
.expect(expectedCount, method(HttpMethod.POST))
|
||||
.andExpect(requestTo("http://localhost:9000/mtbfile/MTBFile"))
|
||||
.andRespond {
|
||||
withStatus(requestWithResponse.httpStatus).body(requestWithResponse.body).createResponse(it)
|
||||
}
|
||||
|
||||
val response = restMtbFileSender.send(MtbFileSender.MtbFileRequest("TestID", mtbFile))
|
||||
val response = restMtbFileSender.send(BwhcV1MtbFileRequest(TEST_REQUEST_ID, mtbFile))
|
||||
assertThat(response.status).isEqualTo(requestWithResponse.response.status)
|
||||
assertThat(response.body).isEqualTo(requestWithResponse.response.body)
|
||||
}
|
||||
@ -117,11 +128,12 @@ class RestMtbFileSenderTest {
|
||||
@MethodSource("deleteRequestWithResponseSource")
|
||||
fun shouldRetryOnDeleteHttpRequestError(requestWithResponse: RequestWithResponse) {
|
||||
val restTemplate = RestTemplate()
|
||||
val restTargetProperties = RestTargetProperties("http://localhost:9000/mtbfile")
|
||||
val restTargetProperties = RestTargetProperties("http://localhost:9000/mtbfile", null, null)
|
||||
val retryTemplate = RetryTemplateBuilder().customPolicy(SimpleRetryPolicy(3)).build()
|
||||
|
||||
this.mockRestServiceServer = MockRestServiceServer.createServer(restTemplate)
|
||||
this.restMtbFileSender = RestMtbFileSender(restTemplate, restTargetProperties, retryTemplate)
|
||||
this.restMtbFileSender =
|
||||
RestBwhcMtbFileSender(restTemplate, restTargetProperties, retryTemplate, reportService)
|
||||
|
||||
val expectedCount = when (requestWithResponse.httpStatus) {
|
||||
// OK - No Retry
|
||||
@ -130,14 +142,14 @@ class RestMtbFileSenderTest {
|
||||
else -> ExpectedCount.max(3)
|
||||
}
|
||||
|
||||
this.mockRestServiceServer.expect(expectedCount) {
|
||||
method(HttpMethod.DELETE)
|
||||
requestTo("/mtbfile")
|
||||
}.andRespond {
|
||||
this.mockRestServiceServer
|
||||
.expect(expectedCount, method(HttpMethod.DELETE))
|
||||
.andExpect(requestTo("http://localhost:9000/mtbfile/Patient/${TEST_PATIENT_PSEUDONYM.value}"))
|
||||
.andRespond {
|
||||
withStatus(requestWithResponse.httpStatus).body(requestWithResponse.body).createResponse(it)
|
||||
}
|
||||
|
||||
val response = restMtbFileSender.send(MtbFileSender.DeleteRequest("TestID", "PID"))
|
||||
val response = restMtbFileSender.send(DeleteRequest(TEST_REQUEST_ID, TEST_PATIENT_PSEUDONYM))
|
||||
assertThat(response.status).isEqualTo(requestWithResponse.response.status)
|
||||
assertThat(response.body).isEqualTo(requestWithResponse.response.body)
|
||||
}
|
||||
@ -149,23 +161,8 @@ class RestMtbFileSenderTest {
|
||||
val response: MtbFileSender.Response
|
||||
)
|
||||
|
||||
private val warningBody = """
|
||||
{
|
||||
"patient_id": "PID",
|
||||
"issues": [
|
||||
{ "severity": "warning", "message": "Something is not right" }
|
||||
]
|
||||
}
|
||||
""".trimIndent()
|
||||
|
||||
private val errorBody = """
|
||||
{
|
||||
"patient_id": "PID",
|
||||
"issues": [
|
||||
{ "severity": "error", "message": "Something is very bad" }
|
||||
]
|
||||
}
|
||||
""".trimIndent()
|
||||
val TEST_REQUEST_ID = RequestId("TestId")
|
||||
val TEST_PATIENT_PSEUDONYM = PatientPseudonym("PID")
|
||||
|
||||
val mtbFile: MtbFile = MtbFile.builder()
|
||||
.withPatient(
|
||||
@ -200,31 +197,44 @@ class RestMtbFileSenderTest {
|
||||
@JvmStatic
|
||||
fun mtbFileRequestWithResponseSource(): Set<RequestWithResponse> {
|
||||
return setOf(
|
||||
RequestWithResponse(HttpStatus.OK, "{}", MtbFileSender.Response(RequestStatus.SUCCESS, "{}")),
|
||||
RequestWithResponse(
|
||||
HttpStatus.OK,
|
||||
responseBodyWithMaxSeverity(ReportService.Severity.INFO),
|
||||
MtbFileSender.Response(
|
||||
RequestStatus.SUCCESS,
|
||||
responseBodyWithMaxSeverity(ReportService.Severity.INFO)
|
||||
)
|
||||
),
|
||||
RequestWithResponse(
|
||||
HttpStatus.CREATED,
|
||||
warningBody,
|
||||
MtbFileSender.Response(RequestStatus.WARNING, warningBody)
|
||||
responseBodyWithMaxSeverity(ReportService.Severity.WARNING),
|
||||
MtbFileSender.Response(
|
||||
RequestStatus.WARNING,
|
||||
responseBodyWithMaxSeverity(ReportService.Severity.WARNING)
|
||||
)
|
||||
),
|
||||
RequestWithResponse(
|
||||
HttpStatus.BAD_REQUEST,
|
||||
"??",
|
||||
MtbFileSender.Response(RequestStatus.ERROR, ERROR_RESPONSE_BODY)
|
||||
responseBodyWithMaxSeverity(ReportService.Severity.ERROR),
|
||||
MtbFileSender.Response(RequestStatus.ERROR, responseBodyWithMaxSeverity(ReportService.Severity.ERROR))
|
||||
),
|
||||
RequestWithResponse(
|
||||
HttpStatus.UNPROCESSABLE_ENTITY,
|
||||
errorBody,
|
||||
MtbFileSender.Response(RequestStatus.ERROR, ERROR_RESPONSE_BODY)
|
||||
responseBodyWithMaxSeverity(ReportService.Severity.FATAL),
|
||||
MtbFileSender.Response(
|
||||
RequestStatus.ERROR,
|
||||
responseBodyWithMaxSeverity(ReportService.Severity.FATAL)
|
||||
)
|
||||
),
|
||||
// Some more errors not mentioned in documentation
|
||||
RequestWithResponse(
|
||||
HttpStatus.NOT_FOUND,
|
||||
"what????",
|
||||
ERROR_RESPONSE_BODY,
|
||||
MtbFileSender.Response(RequestStatus.ERROR, ERROR_RESPONSE_BODY)
|
||||
),
|
||||
RequestWithResponse(
|
||||
HttpStatus.INTERNAL_SERVER_ERROR,
|
||||
"what????",
|
||||
ERROR_RESPONSE_BODY,
|
||||
MtbFileSender.Response(RequestStatus.ERROR, ERROR_RESPONSE_BODY)
|
||||
)
|
||||
)
|
||||
@ -251,6 +261,52 @@ class RestMtbFileSenderTest {
|
||||
)
|
||||
)
|
||||
}
|
||||
|
||||
fun responseBodyWithMaxSeverity(severity: ReportService.Severity): String {
|
||||
return when (severity) {
|
||||
ReportService.Severity.INFO -> """
|
||||
{
|
||||
"patient": "PID",
|
||||
"issues": [
|
||||
{ "severity": "info", "message": "Info Message" }
|
||||
]
|
||||
}
|
||||
"""
|
||||
|
||||
ReportService.Severity.WARNING -> """
|
||||
{
|
||||
"patient": "PID",
|
||||
"issues": [
|
||||
{ "severity": "info", "message": "Info Message" },
|
||||
{ "severity": "warning", "message": "Warning Message" }
|
||||
]
|
||||
}
|
||||
"""
|
||||
|
||||
ReportService.Severity.ERROR -> """
|
||||
{
|
||||
"patient": "PID",
|
||||
"issues": [
|
||||
{ "severity": "info", "message": "Info Message" },
|
||||
{ "severity": "warning", "message": "Warning Message" },
|
||||
{ "severity": "error", "message": "Error Message" }
|
||||
]
|
||||
}
|
||||
"""
|
||||
|
||||
ReportService.Severity.FATAL -> """
|
||||
{
|
||||
"patient": "PID",
|
||||
"issues": [
|
||||
{ "severity": "info", "message": "Info Message" },
|
||||
{ "severity": "warning", "message": "Warning Message" },
|
||||
{ "severity": "error", "message": "Error Message" },
|
||||
{ "severity": "fatal", "message": "Fatal Message" }
|
||||
]
|
||||
}
|
||||
"""
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
|
@ -0,0 +1,405 @@
|
||||
/*
|
||||
* This file is part of ETL-Processor
|
||||
*
|
||||
* Copyright (c) 2025 Comprehensive Cancer Center Mainfranken, Datenintegrationszentrum Philipps-Universität Marburg and Contributors
|
||||
*
|
||||
* This program is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
* by the Free Software Foundation, either version 3 of the License, or
|
||||
* (at your option) any later version.
|
||||
*
|
||||
* This program is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
* GNU Affero General Public License for more details.
|
||||
*
|
||||
* You should have received a copy of the GNU Affero General Public License
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
*/
|
||||
|
||||
package dev.dnpm.etl.processor.output
|
||||
|
||||
import com.fasterxml.jackson.databind.ObjectMapper
|
||||
import com.fasterxml.jackson.module.kotlin.KotlinModule
|
||||
import de.ukw.ccc.bwhc.dto.*
|
||||
import de.ukw.ccc.bwhc.dto.Patient
|
||||
import dev.dnpm.etl.processor.CustomMediaType
|
||||
import dev.dnpm.etl.processor.PatientPseudonym
|
||||
import dev.dnpm.etl.processor.RequestId
|
||||
import dev.dnpm.etl.processor.config.AppConfigProperties
|
||||
import dev.dnpm.etl.processor.config.AppConfiguration
|
||||
import dev.dnpm.etl.processor.config.RestTargetProperties
|
||||
import dev.dnpm.etl.processor.monitoring.ReportService
|
||||
import dev.dnpm.etl.processor.monitoring.RequestStatus
|
||||
import dev.pcvolkmer.mv64e.mtb.*
|
||||
import org.assertj.core.api.Assertions.assertThat
|
||||
import org.junit.jupiter.api.BeforeEach
|
||||
import org.junit.jupiter.api.Nested
|
||||
import org.junit.jupiter.params.ParameterizedTest
|
||||
import org.junit.jupiter.params.provider.MethodSource
|
||||
import org.springframework.http.HttpHeaders
|
||||
import org.springframework.http.HttpMethod
|
||||
import org.springframework.http.HttpStatus
|
||||
import org.springframework.http.MediaType
|
||||
import org.springframework.retry.backoff.NoBackOffPolicy
|
||||
import org.springframework.retry.policy.SimpleRetryPolicy
|
||||
import org.springframework.retry.support.RetryTemplateBuilder
|
||||
import org.springframework.test.web.client.ExpectedCount
|
||||
import org.springframework.test.web.client.MockRestServiceServer
|
||||
import org.springframework.test.web.client.match.MockRestRequestMatchers.*
|
||||
import org.springframework.test.web.client.response.MockRestResponseCreators.withStatus
|
||||
import org.springframework.web.client.RestTemplate
|
||||
|
||||
class RestDipMtbFileSenderTest {
|
||||
|
||||
@Nested
|
||||
inner class BwhcV1ContentRequest {
|
||||
|
||||
private lateinit var mockRestServiceServer: MockRestServiceServer
|
||||
|
||||
private lateinit var restMtbFileSender: RestMtbFileSender
|
||||
|
||||
private var reportService = ReportService(ObjectMapper().registerModule(KotlinModule.Builder().build()))
|
||||
|
||||
@BeforeEach
|
||||
fun setup() {
|
||||
val restTemplate = RestTemplate()
|
||||
val restTargetProperties = RestTargetProperties("http://localhost:9000/api", null, null, false)
|
||||
val retryTemplate = RetryTemplateBuilder().customPolicy(SimpleRetryPolicy(1)).build()
|
||||
|
||||
this.mockRestServiceServer = MockRestServiceServer.createServer(restTemplate)
|
||||
|
||||
this.restMtbFileSender =
|
||||
RestDipMtbFileSender(restTemplate, restTargetProperties, retryTemplate, reportService)
|
||||
}
|
||||
|
||||
@ParameterizedTest
|
||||
@MethodSource("dev.dnpm.etl.processor.output.RestDipMtbFileSenderTest#mtbFileRequestWithResponseSource")
|
||||
fun shouldReturnExpectedResponseForMtbFilePost(requestWithResponse: RequestWithResponse) {
|
||||
this.mockRestServiceServer
|
||||
.expect(method(HttpMethod.POST))
|
||||
.andExpect(requestTo("http://localhost:9000/api/mtb/etl/patient-record"))
|
||||
.andExpect(header(HttpHeaders.CONTENT_TYPE, MediaType.APPLICATION_JSON_VALUE))
|
||||
.andRespond {
|
||||
withStatus(requestWithResponse.httpStatus).body(requestWithResponse.body).createResponse(it)
|
||||
}
|
||||
|
||||
val response = restMtbFileSender.send(BwhcV1MtbFileRequest(TEST_REQUEST_ID, bwhcV1mtbFile))
|
||||
assertThat(response.status).isEqualTo(requestWithResponse.response.status)
|
||||
assertThat(response.body).isEqualTo(requestWithResponse.response.body)
|
||||
}
|
||||
|
||||
@ParameterizedTest
|
||||
@MethodSource("dev.dnpm.etl.processor.output.RestDipMtbFileSenderTest#mtbFileRequestWithResponseSource")
|
||||
fun shouldRetryOnMtbFileHttpRequestError(requestWithResponse: RequestWithResponse) {
|
||||
val restTemplate = RestTemplate()
|
||||
val restTargetProperties = RestTargetProperties("http://localhost:9000/api", null, null, false)
|
||||
val retryTemplate = AppConfiguration().retryTemplate(AppConfigProperties("http://localhost:9000"))
|
||||
retryTemplate.setBackOffPolicy(NoBackOffPolicy())
|
||||
|
||||
this.mockRestServiceServer = MockRestServiceServer.createServer(restTemplate)
|
||||
this.restMtbFileSender =
|
||||
RestDipMtbFileSender(restTemplate, restTargetProperties, retryTemplate, reportService)
|
||||
|
||||
val expectedCount = when (requestWithResponse.httpStatus) {
|
||||
// OK - No Retry
|
||||
HttpStatus.OK, HttpStatus.CREATED, HttpStatus.UNPROCESSABLE_ENTITY, HttpStatus.BAD_REQUEST -> ExpectedCount.max(
|
||||
1
|
||||
)
|
||||
// Request failed - Retry max 3 times
|
||||
else -> ExpectedCount.max(3)
|
||||
}
|
||||
|
||||
this.mockRestServiceServer
|
||||
.expect(expectedCount, method(HttpMethod.POST))
|
||||
.andExpect(requestTo("http://localhost:9000/api/mtb/etl/patient-record"))
|
||||
.andRespond {
|
||||
withStatus(requestWithResponse.httpStatus).body(requestWithResponse.body).createResponse(it)
|
||||
}
|
||||
|
||||
val response = restMtbFileSender.send(BwhcV1MtbFileRequest(TEST_REQUEST_ID, bwhcV1mtbFile))
|
||||
assertThat(response.status).isEqualTo(requestWithResponse.response.status)
|
||||
assertThat(response.body).isEqualTo(requestWithResponse.response.body)
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
@Nested
|
||||
inner class DnpmV2ContentRequest {
|
||||
|
||||
private lateinit var mockRestServiceServer: MockRestServiceServer
|
||||
|
||||
private lateinit var restMtbFileSender: RestMtbFileSender
|
||||
|
||||
private var reportService = ReportService(ObjectMapper().registerModule(KotlinModule.Builder().build()))
|
||||
|
||||
@BeforeEach
|
||||
fun setup() {
|
||||
val restTemplate = RestTemplate()
|
||||
val restTargetProperties = RestTargetProperties("http://localhost:9000/api", null, null, false)
|
||||
val retryTemplate = RetryTemplateBuilder().customPolicy(SimpleRetryPolicy(1)).build()
|
||||
|
||||
this.mockRestServiceServer = MockRestServiceServer.createServer(restTemplate)
|
||||
|
||||
this.restMtbFileSender = RestDipMtbFileSender(restTemplate, restTargetProperties, retryTemplate, reportService)
|
||||
}
|
||||
|
||||
@ParameterizedTest
|
||||
@MethodSource("dev.dnpm.etl.processor.output.RestDipMtbFileSenderTest#mtbFileRequestWithResponseSource")
|
||||
fun shouldReturnExpectedResponseForDnpmV2MtbFilePost(requestWithResponse: RequestWithResponse) {
|
||||
this.mockRestServiceServer
|
||||
.expect(method(HttpMethod.POST))
|
||||
.andExpect(requestTo("http://localhost:9000/api/mtb/etl/patient-record"))
|
||||
.andExpect(header(HttpHeaders.CONTENT_TYPE, CustomMediaType.APPLICATION_VND_DNPM_V2_MTB_JSON_VALUE))
|
||||
.andRespond {
|
||||
withStatus(requestWithResponse.httpStatus).body(requestWithResponse.body).createResponse(it)
|
||||
}
|
||||
|
||||
val response = restMtbFileSender.send(DnpmV2MtbFileRequest(TEST_REQUEST_ID, dnpmV2MtbFile))
|
||||
assertThat(response.status).isEqualTo(requestWithResponse.response.status)
|
||||
assertThat(response.body).isEqualTo(requestWithResponse.response.body)
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
@Nested
|
||||
inner class DeleteRequest {
|
||||
|
||||
private lateinit var mockRestServiceServer: MockRestServiceServer
|
||||
|
||||
private lateinit var restMtbFileSender: RestMtbFileSender
|
||||
|
||||
private var reportService = ReportService(ObjectMapper().registerModule(KotlinModule.Builder().build()))
|
||||
|
||||
@BeforeEach
|
||||
fun setup() {
|
||||
val restTemplate = RestTemplate()
|
||||
val restTargetProperties = RestTargetProperties("http://localhost:9000/api", null, null, false)
|
||||
val retryTemplate = RetryTemplateBuilder().customPolicy(SimpleRetryPolicy(1)).build()
|
||||
|
||||
this.mockRestServiceServer = MockRestServiceServer.createServer(restTemplate)
|
||||
|
||||
this.restMtbFileSender =
|
||||
RestDipMtbFileSender(restTemplate, restTargetProperties, retryTemplate, reportService)
|
||||
}
|
||||
|
||||
@ParameterizedTest
|
||||
@MethodSource("dev.dnpm.etl.processor.output.RestDipMtbFileSenderTest#deleteRequestWithResponseSource")
|
||||
fun shouldReturnExpectedResponseForDelete(requestWithResponse: RequestWithResponse) {
|
||||
this.mockRestServiceServer
|
||||
.expect(method(HttpMethod.DELETE))
|
||||
.andExpect(requestTo("http://localhost:9000/api/mtb/etl/patient/${TEST_PATIENT_PSEUDONYM.value}"))
|
||||
.andRespond {
|
||||
withStatus(requestWithResponse.httpStatus).body(requestWithResponse.body).createResponse(it)
|
||||
}
|
||||
|
||||
val response = restMtbFileSender.send(DeleteRequest(TEST_REQUEST_ID, TEST_PATIENT_PSEUDONYM))
|
||||
assertThat(response.status).isEqualTo(requestWithResponse.response.status)
|
||||
assertThat(response.body).isEqualTo(requestWithResponse.response.body)
|
||||
}
|
||||
|
||||
@ParameterizedTest
|
||||
@MethodSource("dev.dnpm.etl.processor.output.RestDipMtbFileSenderTest#deleteRequestWithResponseSource")
|
||||
fun shouldRetryOnDeleteHttpRequestError(requestWithResponse: RequestWithResponse) {
|
||||
val restTemplate = RestTemplate()
|
||||
val restTargetProperties = RestTargetProperties("http://localhost:9000/api", null, null, false)
|
||||
val retryTemplate = AppConfiguration().retryTemplate(AppConfigProperties("http://localhost:9000"))
|
||||
retryTemplate.setBackOffPolicy(NoBackOffPolicy())
|
||||
|
||||
this.mockRestServiceServer = MockRestServiceServer.createServer(restTemplate)
|
||||
this.restMtbFileSender =
|
||||
RestDipMtbFileSender(restTemplate, restTargetProperties, retryTemplate, reportService)
|
||||
|
||||
val expectedCount = when (requestWithResponse.httpStatus) {
|
||||
// OK - No Retry
|
||||
HttpStatus.OK, HttpStatus.CREATED, HttpStatus.UNPROCESSABLE_ENTITY, HttpStatus.BAD_REQUEST -> ExpectedCount.max(
|
||||
1
|
||||
)
|
||||
// Request failed - Retry max 3 times
|
||||
else -> ExpectedCount.max(3)
|
||||
}
|
||||
|
||||
this.mockRestServiceServer
|
||||
.expect(expectedCount, method(HttpMethod.DELETE))
|
||||
.andExpect(requestTo("http://localhost:9000/api/mtb/etl/patient/${TEST_PATIENT_PSEUDONYM.value}"))
|
||||
.andRespond {
|
||||
withStatus(requestWithResponse.httpStatus).body(requestWithResponse.body).createResponse(it)
|
||||
}
|
||||
|
||||
val response = restMtbFileSender.send(DeleteRequest(TEST_REQUEST_ID, TEST_PATIENT_PSEUDONYM))
|
||||
assertThat(response.status).isEqualTo(requestWithResponse.response.status)
|
||||
assertThat(response.body).isEqualTo(requestWithResponse.response.body)
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
companion object {
|
||||
data class RequestWithResponse(
|
||||
val httpStatus: HttpStatus,
|
||||
val body: String,
|
||||
val response: MtbFileSender.Response
|
||||
)
|
||||
|
||||
val TEST_REQUEST_ID = RequestId("TestId")
|
||||
val TEST_PATIENT_PSEUDONYM = PatientPseudonym("PID")
|
||||
|
||||
val bwhcV1mtbFile: MtbFile = MtbFile.builder()
|
||||
.withPatient(
|
||||
Patient.builder()
|
||||
.withId("PID")
|
||||
.withBirthDate("2000-08-08")
|
||||
.withGender(Patient.Gender.MALE)
|
||||
.build()
|
||||
)
|
||||
.withConsent(
|
||||
Consent.builder()
|
||||
.withId("1")
|
||||
.withStatus(Consent.Status.ACTIVE)
|
||||
.withPatient("PID")
|
||||
.build()
|
||||
)
|
||||
.withEpisode(
|
||||
Episode.builder()
|
||||
.withId("1")
|
||||
.withPatient("PID")
|
||||
.withPeriod(PeriodStart("2023-08-08"))
|
||||
.build()
|
||||
)
|
||||
.build()
|
||||
|
||||
val dnpmV2MtbFile: Mtb = Mtb.builder()
|
||||
.withPatient(
|
||||
dev.pcvolkmer.mv64e.mtb.Patient.builder()
|
||||
.withId("PID")
|
||||
.withBirthDate("2000-08-08")
|
||||
.withGender(CodingGender.builder().withCode(CodingGender.Code.MALE).build())
|
||||
.build()
|
||||
)
|
||||
.withEpisodesOfCare(
|
||||
listOf(
|
||||
MTBEpisodeOfCare.builder()
|
||||
.withId("1")
|
||||
.withPatient(Reference("PID"))
|
||||
.withPeriod(PeriodDate.builder().withStart("2023-08-08").build())
|
||||
.build()
|
||||
)
|
||||
)
|
||||
.build()
|
||||
|
||||
private const val ERROR_RESPONSE_BODY = "Sonstiger Fehler bei der Übertragung"
|
||||
|
||||
/**
|
||||
* Synthetic http responses with related request status
|
||||
* Also see: https://ibmi-intra.cs.uni-tuebingen.de/display/ZPM/bwHC+REST+API
|
||||
*/
|
||||
@JvmStatic
|
||||
fun mtbFileRequestWithResponseSource(): Set<RequestWithResponse> {
|
||||
return setOf(
|
||||
RequestWithResponse(
|
||||
HttpStatus.OK,
|
||||
responseBodyWithMaxSeverity(ReportService.Severity.INFO),
|
||||
MtbFileSender.Response(
|
||||
RequestStatus.SUCCESS,
|
||||
responseBodyWithMaxSeverity(ReportService.Severity.INFO)
|
||||
)
|
||||
),
|
||||
RequestWithResponse(
|
||||
HttpStatus.CREATED,
|
||||
responseBodyWithMaxSeverity(ReportService.Severity.WARNING),
|
||||
MtbFileSender.Response(RequestStatus.WARNING, responseBodyWithMaxSeverity(ReportService.Severity.WARNING))
|
||||
),
|
||||
RequestWithResponse(
|
||||
HttpStatus.BAD_REQUEST,
|
||||
responseBodyWithMaxSeverity(ReportService.Severity.ERROR),
|
||||
MtbFileSender.Response(RequestStatus.ERROR, responseBodyWithMaxSeverity(ReportService.Severity.ERROR))
|
||||
),
|
||||
RequestWithResponse(
|
||||
HttpStatus.UNPROCESSABLE_ENTITY,
|
||||
responseBodyWithMaxSeverity(ReportService.Severity.ERROR),
|
||||
MtbFileSender.Response(RequestStatus.ERROR, responseBodyWithMaxSeverity(ReportService.Severity.ERROR))
|
||||
),
|
||||
// Some more errors not mentioned in documentation
|
||||
RequestWithResponse(
|
||||
HttpStatus.NOT_FOUND,
|
||||
ERROR_RESPONSE_BODY,
|
||||
MtbFileSender.Response(RequestStatus.ERROR, ERROR_RESPONSE_BODY)
|
||||
),
|
||||
RequestWithResponse(
|
||||
HttpStatus.INTERNAL_SERVER_ERROR,
|
||||
ERROR_RESPONSE_BODY,
|
||||
MtbFileSender.Response(RequestStatus.ERROR, ERROR_RESPONSE_BODY)
|
||||
)
|
||||
)
|
||||
}
|
||||
|
||||
/**
|
||||
* Synthetic http responses with related request status
|
||||
* Also see: https://ibmi-intra.cs.uni-tuebingen.de/display/ZPM/bwHC+REST+API
|
||||
*/
|
||||
@JvmStatic
|
||||
fun deleteRequestWithResponseSource(): Set<RequestWithResponse> {
|
||||
return setOf(
|
||||
RequestWithResponse(HttpStatus.OK, "", MtbFileSender.Response(RequestStatus.SUCCESS)),
|
||||
// Some more errors not mentioned in documentation
|
||||
RequestWithResponse(
|
||||
HttpStatus.NOT_FOUND,
|
||||
"what????",
|
||||
MtbFileSender.Response(RequestStatus.ERROR, ERROR_RESPONSE_BODY)
|
||||
),
|
||||
RequestWithResponse(
|
||||
HttpStatus.INTERNAL_SERVER_ERROR,
|
||||
"what????",
|
||||
MtbFileSender.Response(RequestStatus.ERROR, ERROR_RESPONSE_BODY)
|
||||
)
|
||||
)
|
||||
}
|
||||
|
||||
fun responseBodyWithMaxSeverity(severity: ReportService.Severity): String {
|
||||
return when (severity) {
|
||||
ReportService.Severity.INFO -> """
|
||||
{
|
||||
"patient": "PID",
|
||||
"issues": [
|
||||
{ "severity": "info", "message": "Info Message" }
|
||||
]
|
||||
}
|
||||
"""
|
||||
|
||||
ReportService.Severity.WARNING -> """
|
||||
{
|
||||
"patient": "PID",
|
||||
"issues": [
|
||||
{ "severity": "info", "message": "Info Message" },
|
||||
{ "severity": "warning", "message": "Warning Message" }
|
||||
]
|
||||
}
|
||||
"""
|
||||
|
||||
ReportService.Severity.ERROR -> """
|
||||
{
|
||||
"patient": "PID",
|
||||
"issues": [
|
||||
{ "severity": "info", "message": "Info Message" },
|
||||
{ "severity": "warning", "message": "Warning Message" },
|
||||
{ "severity": "error", "message": "Error Message" }
|
||||
]
|
||||
}
|
||||
"""
|
||||
|
||||
ReportService.Severity.FATAL -> """
|
||||
{
|
||||
"patient": "PID",
|
||||
"issues": [
|
||||
{ "severity": "info", "message": "Info Message" },
|
||||
{ "severity": "warning", "message": "Warning Message" },
|
||||
{ "severity": "error", "message": "Error Message" },
|
||||
{ "severity": "fatal", "message": "Fatal Message" }
|
||||
]
|
||||
}
|
||||
"""
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
}
|
@ -1,7 +1,7 @@
|
||||
/*
|
||||
* This file is part of ETL-Processor
|
||||
*
|
||||
* Copyright (c) 2023 Comprehensive Cancer Center Mainfranken, Datenintegrationszentrum Philipps-Universität Marburg and Contributors
|
||||
* Copyright (c) 2025 Comprehensive Cancer Center Mainfranken, Datenintegrationszentrum Philipps-Universität Marburg and Contributors
|
||||
*
|
||||
* This program is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
@ -21,23 +21,31 @@ package dev.dnpm.etl.processor.pseudonym
|
||||
|
||||
import com.fasterxml.jackson.databind.ObjectMapper
|
||||
import de.ukw.ccc.bwhc.dto.*
|
||||
import dev.pcvolkmer.mv64e.mtb.MTBEpisodeOfCare
|
||||
import dev.pcvolkmer.mv64e.mtb.Mtb
|
||||
import dev.pcvolkmer.mv64e.mtb.PeriodDate
|
||||
import dev.pcvolkmer.mv64e.mtb.Reference
|
||||
import org.assertj.core.api.Assertions.assertThat
|
||||
import org.junit.jupiter.api.Nested
|
||||
import org.junit.jupiter.api.Test
|
||||
import org.junit.jupiter.api.assertThrows
|
||||
import org.junit.jupiter.api.extension.ExtendWith
|
||||
import org.mockito.ArgumentMatchers
|
||||
import org.mockito.Mock
|
||||
import org.mockito.junit.jupiter.MockitoExtension
|
||||
import org.mockito.kotlin.anyValueClass
|
||||
import org.mockito.kotlin.doAnswer
|
||||
import org.mockito.kotlin.whenever
|
||||
import org.springframework.core.io.ClassPathResource
|
||||
|
||||
const val FAKE_MTB_FILE_PATH = "fake_MTBFile.json"
|
||||
const val CLEAN_PATIENT_ID = "5dad2f0b-49c6-47d8-a952-7b9e9e0f7549"
|
||||
|
||||
@ExtendWith(MockitoExtension::class)
|
||||
class ExtensionsTest {
|
||||
|
||||
@Nested
|
||||
inner class UsingBwhcDatamodel {
|
||||
|
||||
val FAKE_MTB_FILE_PATH = "fake_MTBFile.json"
|
||||
val CLEAN_PATIENT_ID = "5dad2f0b-49c6-47d8-a952-7b9e9e0f7549"
|
||||
|
||||
private fun fakeMtbFile(): MtbFile {
|
||||
val mtbFile = ClassPathResource(FAKE_MTB_FILE_PATH).inputStream
|
||||
return ObjectMapper().readValue(mtbFile, MtbFile::class.java)
|
||||
@ -52,7 +60,7 @@ class ExtensionsTest {
|
||||
doAnswer {
|
||||
it.arguments[0]
|
||||
"PSEUDO-ID"
|
||||
}.whenever(pseudonymizeService).patientPseudonym(ArgumentMatchers.anyString())
|
||||
}.whenever(pseudonymizeService).patientPseudonym(anyValueClass())
|
||||
|
||||
val mtbFile = fakeMtbFile()
|
||||
|
||||
@ -67,7 +75,7 @@ class ExtensionsTest {
|
||||
doAnswer {
|
||||
it.arguments[0]
|
||||
"PSEUDO-ID"
|
||||
}.whenever(pseudonymizeService).patientPseudonym(ArgumentMatchers.anyString())
|
||||
}.whenever(pseudonymizeService).patientPseudonym(anyValueClass())
|
||||
|
||||
doAnswer {
|
||||
"TESTDOMAIN"
|
||||
@ -95,7 +103,7 @@ class ExtensionsTest {
|
||||
doAnswer {
|
||||
it.arguments[0]
|
||||
"PSEUDO-ID"
|
||||
}.whenever(pseudonymizeService).patientPseudonym(ArgumentMatchers.anyString())
|
||||
}.whenever(pseudonymizeService).patientPseudonym(anyValueClass())
|
||||
|
||||
doAnswer {
|
||||
"TESTDOMAIN"
|
||||
@ -139,7 +147,7 @@ class ExtensionsTest {
|
||||
doAnswer {
|
||||
it.arguments[0]
|
||||
"PSEUDO-ID"
|
||||
}.whenever(pseudonymizeService).patientPseudonym(ArgumentMatchers.anyString())
|
||||
}.whenever(pseudonymizeService).patientPseudonym(anyValueClass())
|
||||
|
||||
doAnswer {
|
||||
"TESTDOMAIN"
|
||||
@ -191,8 +199,83 @@ class ExtensionsTest {
|
||||
mtbFile.pseudonymizeWith(pseudonymizeService)
|
||||
mtbFile.anonymizeContentWith(pseudonymizeService)
|
||||
|
||||
|
||||
assertThat(mtbFile.episode.id).isNotNull()
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
@Nested
|
||||
inner class UsingDnpmV2Datamodel {
|
||||
|
||||
val FAKE_MTB_FILE_PATH = "mv64e-mtb-fake-patient.json"
|
||||
val CLEAN_PATIENT_ID = "63f8fd7b-8127-4f3c-8843-aa9199e21c29"
|
||||
|
||||
private fun fakeMtbFile(): Mtb {
|
||||
val mtbFile = ClassPathResource(FAKE_MTB_FILE_PATH).inputStream
|
||||
return ObjectMapper().readValue(mtbFile, Mtb::class.java)
|
||||
}
|
||||
|
||||
private fun Mtb.serialized(): String {
|
||||
return ObjectMapper().writeValueAsString(this)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun shouldNotContainCleanPatientId(@Mock pseudonymizeService: PseudonymizeService) {
|
||||
doAnswer {
|
||||
it.arguments[0]
|
||||
"PSEUDO-ID"
|
||||
}.whenever(pseudonymizeService).patientPseudonym(anyValueClass())
|
||||
|
||||
val mtbFile = fakeMtbFile()
|
||||
|
||||
mtbFile.pseudonymizeWith(pseudonymizeService)
|
||||
|
||||
assertThat(mtbFile.patient.id).isEqualTo("PSEUDO-ID")
|
||||
assertThat(mtbFile.serialized()).doesNotContain(CLEAN_PATIENT_ID)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun shouldNotThrowExceptionOnNullValues(@Mock pseudonymizeService: PseudonymizeService) {
|
||||
doAnswer {
|
||||
it.arguments[0]
|
||||
"PSEUDO-ID"
|
||||
}.whenever(pseudonymizeService).patientPseudonym(anyValueClass())
|
||||
|
||||
doAnswer {
|
||||
"TESTDOMAIN"
|
||||
}.whenever(pseudonymizeService).prefix()
|
||||
|
||||
val mtbFile = Mtb.builder()
|
||||
.withPatient(
|
||||
dev.pcvolkmer.mv64e.mtb.Patient.builder()
|
||||
.withId("1")
|
||||
.withBirthDate("2000-08-08")
|
||||
.withGender(null)
|
||||
.build()
|
||||
)
|
||||
.withEpisodesOfCare(
|
||||
listOf(
|
||||
MTBEpisodeOfCare.builder()
|
||||
.withId("1")
|
||||
.withPatient(Reference("1"))
|
||||
.withPeriod(PeriodDate.builder().withStart("2023-08-08").build())
|
||||
.build()
|
||||
)
|
||||
)
|
||||
.withClaims(null)
|
||||
.withDiagnoses(null)
|
||||
.withCarePlans(null)
|
||||
.withClaimResponses(null)
|
||||
.withHistologyReports(null)
|
||||
.withNgsReports(null)
|
||||
.withResponses(null)
|
||||
.withSpecimens(null)
|
||||
.build()
|
||||
|
||||
mtbFile.pseudonymizeWith(pseudonymizeService)
|
||||
mtbFile.anonymizeContentWith(pseudonymizeService)
|
||||
|
||||
assertThat(mtbFile.episodesOfCare).hasSize(1)
|
||||
assertThat(mtbFile.episodesOfCare.map { it.id }).isNotNull
|
||||
}
|
||||
}
|
||||
}
|
@ -17,13 +17,12 @@
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
*/
|
||||
|
||||
package dev.dnpm.etl.processor.services
|
||||
package dev.dnpm.etl.processor.security
|
||||
|
||||
import org.assertj.core.api.Assertions.assertThat
|
||||
import org.junit.jupiter.api.BeforeEach
|
||||
import org.junit.jupiter.api.Test
|
||||
import org.junit.jupiter.api.extension.ExtendWith
|
||||
import org.mockito.ArgumentCaptor
|
||||
import org.mockito.ArgumentMatchers.anyLong
|
||||
import org.mockito.ArgumentMatchers.anyString
|
||||
import org.mockito.Mock
|
||||
@ -96,11 +95,11 @@ class TokenServiceTest {
|
||||
|
||||
val actual = this.tokenService.addToken("Test Token")
|
||||
|
||||
val captor = ArgumentCaptor.forClass(Token::class.java)
|
||||
val captor = argumentCaptor<Token>()
|
||||
verify(tokenRepository, times(1)).save(captor.capture())
|
||||
|
||||
assertThat(actual).satisfies(Consumer { assertThat(it.isSuccess).isTrue() })
|
||||
assertThat(captor.value).satisfies(
|
||||
assertThat(captor.firstValue).satisfies(
|
||||
Consumer { assertThat(it.name).isEqualTo("Test Token") },
|
||||
Consumer { assertThat(it.username).isEqualTo("testtoken") },
|
||||
Consumer { assertThat(it.password).isEqualTo("{test}verysecret") }
|
||||
@ -116,13 +115,13 @@ class TokenServiceTest {
|
||||
|
||||
this.tokenService.deleteToken(42)
|
||||
|
||||
val stringCaptor = ArgumentCaptor.forClass(String::class.java)
|
||||
val stringCaptor = argumentCaptor<String>()
|
||||
verify(userDetailsManager, times(1)).deleteUser(stringCaptor.capture())
|
||||
assertThat(stringCaptor.value).isEqualTo("testtoken")
|
||||
assertThat(stringCaptor.firstValue).isEqualTo("testtoken")
|
||||
|
||||
val tokenCaptor = ArgumentCaptor.forClass(Token::class.java)
|
||||
val tokenCaptor = argumentCaptor<Token>()
|
||||
verify(tokenRepository, times(1)).delete(tokenCaptor.capture())
|
||||
assertThat(tokenCaptor.value.id).isEqualTo(42)
|
||||
assertThat(tokenCaptor.firstValue.id).isEqualTo(42)
|
||||
}
|
||||
|
||||
@Test
|
@ -0,0 +1,202 @@
|
||||
/*
|
||||
* This file is part of ETL-Processor
|
||||
*
|
||||
* Copyright (c) 2024 Comprehensive Cancer Center Mainfranken, Datenintegrationszentrum Philipps-Universität Marburg and Contributors
|
||||
*
|
||||
* This program is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
* by the Free Software Foundation, either version 3 of the License, or
|
||||
* (at your option) any later version.
|
||||
*
|
||||
* This program is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
* GNU Affero General Public License for more details.
|
||||
*
|
||||
* You should have received a copy of the GNU Affero General Public License
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
*/
|
||||
|
||||
package dev.dnpm.etl.processor.security
|
||||
|
||||
import org.assertj.core.api.Assertions.assertThat
|
||||
import org.junit.jupiter.api.BeforeEach
|
||||
import org.junit.jupiter.api.Nested
|
||||
import org.junit.jupiter.api.Test
|
||||
import org.junit.jupiter.api.extension.ExtendWith
|
||||
import org.mockito.Mock
|
||||
import org.mockito.junit.jupiter.MockitoExtension
|
||||
import org.mockito.kotlin.*
|
||||
import org.springframework.security.core.session.SessionInformation
|
||||
import org.springframework.security.core.session.SessionRegistry
|
||||
import org.springframework.security.oauth2.core.oidc.OidcIdToken
|
||||
import org.springframework.security.oauth2.core.oidc.user.DefaultOidcUser
|
||||
import org.springframework.security.oauth2.core.oidc.user.OidcUser
|
||||
import java.time.Instant
|
||||
import java.util.*
|
||||
|
||||
@ExtendWith(MockitoExtension::class)
|
||||
class UserRoleServiceTest {
|
||||
|
||||
private lateinit var userRoleRepository: UserRoleRepository
|
||||
private lateinit var sessionRegistry: SessionRegistry
|
||||
|
||||
private lateinit var userRoleService: UserRoleService
|
||||
|
||||
@BeforeEach
|
||||
fun setup(
|
||||
@Mock userRoleRepository: UserRoleRepository,
|
||||
@Mock sessionRegistry: SessionRegistry
|
||||
) {
|
||||
this.userRoleRepository = userRoleRepository
|
||||
this.sessionRegistry = sessionRegistry
|
||||
|
||||
this.userRoleService = UserRoleService(userRoleRepository, sessionRegistry)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun shouldDelegateFindAllToRepository() {
|
||||
userRoleService.findAll()
|
||||
|
||||
verify(userRoleRepository, times(1)).findAll()
|
||||
}
|
||||
|
||||
@Nested
|
||||
inner class WithExistingUserRole {
|
||||
|
||||
@BeforeEach
|
||||
fun setup() {
|
||||
doAnswer { invocation ->
|
||||
Optional.of(
|
||||
UserRole(invocation.getArgument(0), "patrick.tester", Role.USER)
|
||||
)
|
||||
}.whenever(userRoleRepository).findById(any<Long>())
|
||||
|
||||
doAnswer { _ ->
|
||||
listOf(
|
||||
dummyPrincipal()
|
||||
)
|
||||
}.whenever(sessionRegistry).allPrincipals
|
||||
}
|
||||
|
||||
@Test
|
||||
fun shouldUpdateUserRole() {
|
||||
userRoleService.updateUserRole(1, Role.ADMIN)
|
||||
|
||||
val userRoleCaptor = argumentCaptor<UserRole>()
|
||||
verify(userRoleRepository, times(1)).save(userRoleCaptor.capture())
|
||||
|
||||
assertThat(userRoleCaptor.firstValue.id).isEqualTo(1L)
|
||||
assertThat(userRoleCaptor.firstValue.role).isEqualTo(Role.ADMIN)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun shouldExpireSessionOnUpdate() {
|
||||
val dummySessions = dummySessions()
|
||||
whenever(sessionRegistry.getAllSessions(any(), any<Boolean>())).thenReturn(
|
||||
dummySessions
|
||||
)
|
||||
|
||||
assertThat(dummySessions.filter { it.isExpired }).hasSize(0)
|
||||
|
||||
userRoleService.updateUserRole(1, Role.ADMIN)
|
||||
|
||||
verify(sessionRegistry, times(1)).getAllSessions(any<OidcUser>(), any<Boolean>())
|
||||
|
||||
assertThat(dummySessions.filter { it.isExpired }).hasSize(2)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun shouldDeleteUserRole() {
|
||||
userRoleService.deleteUserRole(1)
|
||||
|
||||
val userRoleCaptor = argumentCaptor<UserRole>()
|
||||
verify(userRoleRepository, times(1)).delete(userRoleCaptor.capture())
|
||||
|
||||
assertThat(userRoleCaptor.firstValue.id).isEqualTo(1L)
|
||||
assertThat(userRoleCaptor.firstValue.role).isEqualTo(Role.USER)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun shouldExpireSessionOnDelete() {
|
||||
val dummySessions = dummySessions()
|
||||
whenever(sessionRegistry.getAllSessions(any(), any<Boolean>())).thenReturn(
|
||||
dummySessions
|
||||
)
|
||||
|
||||
assertThat(dummySessions.filter { it.isExpired }).hasSize(0)
|
||||
|
||||
userRoleService.deleteUserRole(1)
|
||||
|
||||
verify(sessionRegistry, times(1)).getAllSessions(any<OidcUser>(), any<Boolean>())
|
||||
|
||||
assertThat(dummySessions.filter { it.isExpired }).hasSize(2)
|
||||
}
|
||||
}
|
||||
|
||||
@Nested
|
||||
inner class WithoutExistingUserRole {
|
||||
|
||||
@BeforeEach
|
||||
fun setup() {
|
||||
doAnswer { _ ->
|
||||
Optional.empty<UserRole>()
|
||||
}.whenever(userRoleRepository).findById(any<Long>())
|
||||
}
|
||||
|
||||
@Test
|
||||
fun shouldNotUpdateUserRole() {
|
||||
userRoleService.updateUserRole(1, Role.ADMIN)
|
||||
|
||||
verify(userRoleRepository, never()).save(any<UserRole>())
|
||||
}
|
||||
|
||||
@Test
|
||||
fun shouldNotExpireSessionOnUpdate() {
|
||||
userRoleService.updateUserRole(1, Role.ADMIN)
|
||||
|
||||
verify(sessionRegistry, never()).getAllSessions(any<OidcUser>(), any<Boolean>())
|
||||
}
|
||||
|
||||
@Test
|
||||
fun shouldNotDeleteUserRole() {
|
||||
userRoleService.deleteUserRole(1)
|
||||
|
||||
verify(userRoleRepository, never()).delete(any<UserRole>())
|
||||
}
|
||||
|
||||
@Test
|
||||
fun shouldNotExpireSessionOnDelete() {
|
||||
userRoleService.deleteUserRole(1)
|
||||
|
||||
verify(sessionRegistry, never()).getAllSessions(any<OidcUser>(), any<Boolean>())
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
|
||||
companion object {
|
||||
private fun dummyPrincipal() = DefaultOidcUser(
|
||||
listOf(),
|
||||
OidcIdToken(
|
||||
"anytokenvalue",
|
||||
Instant.now(),
|
||||
Instant.now().plusSeconds(10),
|
||||
mapOf("sub" to "testsub", "preferred_username" to "patrick.tester")
|
||||
)
|
||||
)
|
||||
|
||||
private fun dummySessions() = listOf(
|
||||
SessionInformation(
|
||||
dummyPrincipal(),
|
||||
"SESSIONID1",
|
||||
Date.from(Instant.now()),
|
||||
),
|
||||
SessionInformation(
|
||||
dummyPrincipal(),
|
||||
"SESSIONID2",
|
||||
Date.from(Instant.now()),
|
||||
)
|
||||
)
|
||||
}
|
||||
}
|
@ -1,7 +1,7 @@
|
||||
/*
|
||||
* This file is part of ETL-Processor
|
||||
*
|
||||
* Copyright (c) 2023 Comprehensive Cancer Center Mainfranken, Datenintegrationszentrum Philipps-Universität Marburg and Contributors
|
||||
* Copyright (c) 2025 Comprehensive Cancer Center Mainfranken, Datenintegrationszentrum Philipps-Universität Marburg and Contributors
|
||||
*
|
||||
* This program is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
@ -22,9 +22,14 @@ package dev.dnpm.etl.processor.services
|
||||
import com.fasterxml.jackson.databind.ObjectMapper
|
||||
import com.fasterxml.jackson.module.kotlin.KotlinModule
|
||||
import dev.dnpm.etl.processor.monitoring.ReportService
|
||||
import dev.dnpm.etl.processor.monitoring.RequestStatus
|
||||
import dev.dnpm.etl.processor.monitoring.asRequestStatus
|
||||
import org.assertj.core.api.Assertions.assertThat
|
||||
import org.junit.jupiter.api.BeforeEach
|
||||
import org.junit.jupiter.api.Test
|
||||
import org.junit.jupiter.params.ParameterizedTest
|
||||
import org.junit.jupiter.params.provider.Arguments
|
||||
import org.junit.jupiter.params.provider.MethodSource
|
||||
|
||||
class ReportServiceTest {
|
||||
|
||||
@ -60,6 +65,15 @@ class ReportServiceTest {
|
||||
assertThat(actual[2].message).isEqualTo("Warning Message")
|
||||
assertThat(actual[3].severity).isEqualTo(ReportService.Severity.INFO)
|
||||
assertThat(actual[3].message).isEqualTo("Info Message")
|
||||
|
||||
assertThat(actual.asRequestStatus()).isEqualTo(RequestStatus.ERROR)
|
||||
}
|
||||
|
||||
@ParameterizedTest
|
||||
@MethodSource("testData")
|
||||
fun shouldParseDataQualityReport(json: String, requestStatus: RequestStatus) {
|
||||
val actual = this.reportService.deserialize(json)
|
||||
assertThat(actual.asRequestStatus()).isEqualTo(requestStatus)
|
||||
}
|
||||
|
||||
@Test
|
||||
@ -73,4 +87,75 @@ class ReportServiceTest {
|
||||
assertThat(actual[0].message).isEqualTo("Not parsable data quality report '$invalidResponse'")
|
||||
}
|
||||
|
||||
companion object {
|
||||
|
||||
@JvmStatic
|
||||
fun testData(): Set<Arguments> {
|
||||
return setOf(
|
||||
Arguments.of(
|
||||
"""
|
||||
{
|
||||
"patient": "4711",
|
||||
"issues": [
|
||||
{ "severity": "info", "message": "Info Message" },
|
||||
{ "severity": "warning", "message": "Warning Message" },
|
||||
{ "severity": "error", "message": "Error Message" },
|
||||
{ "severity": "fatal", "message": "Fatal Message" }
|
||||
]
|
||||
}
|
||||
""".trimIndent(),
|
||||
RequestStatus.ERROR
|
||||
),
|
||||
Arguments.of(
|
||||
"""
|
||||
{
|
||||
"patient": "4711",
|
||||
"issues": [
|
||||
{ "severity": "info", "message": "Info Message" },
|
||||
{ "severity": "warning", "message": "Warning Message" },
|
||||
{ "severity": "error", "message": "Error Message" }
|
||||
]
|
||||
}
|
||||
""".trimIndent(),
|
||||
RequestStatus.ERROR
|
||||
),
|
||||
Arguments.of(
|
||||
"""
|
||||
{
|
||||
"patient": "4711",
|
||||
"issues": [
|
||||
{ "severity": "error", "message": "Error Message" }
|
||||
{ "severity": "info", "message": "Info Message" }
|
||||
]
|
||||
}
|
||||
""".trimIndent(),
|
||||
RequestStatus.ERROR
|
||||
),
|
||||
Arguments.of(
|
||||
"""
|
||||
{
|
||||
"patient": "4711",
|
||||
"issues": [
|
||||
{ "severity": "info", "message": "Info Message" },
|
||||
{ "severity": "warning", "message": "Warning Message" }
|
||||
]
|
||||
}
|
||||
""".trimIndent(),
|
||||
RequestStatus.WARNING
|
||||
),
|
||||
Arguments.of(
|
||||
"""
|
||||
{
|
||||
"patient": "4711",
|
||||
"issues": [
|
||||
{ "severity": "info", "message": "Info Message" }
|
||||
]
|
||||
}
|
||||
""".trimIndent(),
|
||||
RequestStatus.SUCCESS
|
||||
)
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
}
|
@ -21,27 +21,32 @@ package dev.dnpm.etl.processor.services
|
||||
|
||||
import com.fasterxml.jackson.databind.ObjectMapper
|
||||
import de.ukw.ccc.bwhc.dto.*
|
||||
import dev.dnpm.etl.processor.Fingerprint
|
||||
import dev.dnpm.etl.processor.PatientId
|
||||
import dev.dnpm.etl.processor.PatientPseudonym
|
||||
import dev.dnpm.etl.processor.config.AppConfigProperties
|
||||
import dev.dnpm.etl.processor.monitoring.Request
|
||||
import dev.dnpm.etl.processor.monitoring.RequestStatus
|
||||
import dev.dnpm.etl.processor.monitoring.RequestType
|
||||
import dev.dnpm.etl.processor.output.BwhcV1MtbFileRequest
|
||||
import dev.dnpm.etl.processor.output.DeleteRequest
|
||||
import dev.dnpm.etl.processor.output.MtbFileSender
|
||||
import dev.dnpm.etl.processor.output.RestMtbFileSender
|
||||
import dev.dnpm.etl.processor.pseudonym.PseudonymizeService
|
||||
import dev.dnpm.etl.processor.randomRequestId
|
||||
import org.assertj.core.api.Assertions.assertThat
|
||||
import org.junit.jupiter.api.BeforeEach
|
||||
import org.junit.jupiter.api.Test
|
||||
import org.junit.jupiter.api.extension.ExtendWith
|
||||
import org.mockito.ArgumentMatchers.anyString
|
||||
import org.mockito.Mock
|
||||
import org.mockito.Mockito.*
|
||||
import org.mockito.junit.jupiter.MockitoExtension
|
||||
import org.mockito.kotlin.any
|
||||
import org.mockito.kotlin.anyValueClass
|
||||
import org.mockito.kotlin.argumentCaptor
|
||||
import org.mockito.kotlin.whenever
|
||||
import org.springframework.context.ApplicationEventPublisher
|
||||
import java.time.Instant
|
||||
import java.util.*
|
||||
|
||||
|
||||
@ExtendWith(MockitoExtension::class)
|
||||
@ -88,28 +93,28 @@ class RequestProcessorTest {
|
||||
fun testShouldSendMtbFileDuplicationAndSaveUnknownRequestStatusAtFirst() {
|
||||
doAnswer {
|
||||
Request(
|
||||
id = 1L,
|
||||
uuid = UUID.randomUUID().toString(),
|
||||
patientId = "TEST_12345678901",
|
||||
pid = "P1",
|
||||
fingerprint = "zdlzv5s5ydmd4ktw2v5piohegc4jcyrm6j66bq6tv2uxuerndmga",
|
||||
type = RequestType.MTB_FILE,
|
||||
status = RequestStatus.SUCCESS,
|
||||
processedAt = Instant.parse("2023-08-08T02:00:00Z")
|
||||
1L,
|
||||
randomRequestId(),
|
||||
PatientPseudonym("TEST_12345678901"),
|
||||
PatientId("P1"),
|
||||
Fingerprint("zdlzv5s5ydmd4ktw2v5piohegc4jcyrm6j66bq6tv2uxuerndmga"),
|
||||
RequestType.MTB_FILE,
|
||||
RequestStatus.SUCCESS,
|
||||
Instant.parse("2023-08-08T02:00:00Z")
|
||||
)
|
||||
}.`when`(requestService).lastMtbFileRequestForPatientPseudonym(anyString())
|
||||
}.whenever(requestService).lastMtbFileRequestForPatientPseudonym(anyValueClass())
|
||||
|
||||
doAnswer {
|
||||
false
|
||||
}.`when`(requestService).isLastRequestWithKnownStatusDeletion(anyString())
|
||||
}.whenever(requestService).isLastRequestWithKnownStatusDeletion(anyValueClass())
|
||||
|
||||
doAnswer {
|
||||
it.arguments[0] as String
|
||||
}.`when`(pseudonymizeService).patientPseudonym(any())
|
||||
}.whenever(pseudonymizeService).patientPseudonym(anyValueClass())
|
||||
|
||||
doAnswer {
|
||||
it.arguments[0]
|
||||
}.whenever(transformationService).transform(any())
|
||||
}.whenever(transformationService).transform(any<MtbFile>())
|
||||
|
||||
val mtbFile = MtbFile.builder()
|
||||
.withPatient(
|
||||
@ -147,28 +152,28 @@ class RequestProcessorTest {
|
||||
fun testShouldDetectMtbFileDuplicationAndSendDuplicationEvent() {
|
||||
doAnswer {
|
||||
Request(
|
||||
id = 1L,
|
||||
uuid = UUID.randomUUID().toString(),
|
||||
patientId = "TEST_12345678901",
|
||||
pid = "P1",
|
||||
fingerprint = "zdlzv5s5ydmd4ktw2v5piohegc4jcyrm6j66bq6tv2uxuerndmga",
|
||||
type = RequestType.MTB_FILE,
|
||||
status = RequestStatus.SUCCESS,
|
||||
processedAt = Instant.parse("2023-08-08T02:00:00Z")
|
||||
1L,
|
||||
randomRequestId(),
|
||||
PatientPseudonym("TEST_12345678901"),
|
||||
PatientId("P1"),
|
||||
Fingerprint("zdlzv5s5ydmd4ktw2v5piohegc4jcyrm6j66bq6tv2uxuerndmga"),
|
||||
RequestType.MTB_FILE,
|
||||
RequestStatus.SUCCESS,
|
||||
Instant.parse("2023-08-08T02:00:00Z")
|
||||
)
|
||||
}.`when`(requestService).lastMtbFileRequestForPatientPseudonym(anyString())
|
||||
}.whenever(requestService).lastMtbFileRequestForPatientPseudonym(anyValueClass())
|
||||
|
||||
doAnswer {
|
||||
false
|
||||
}.`when`(requestService).isLastRequestWithKnownStatusDeletion(anyString())
|
||||
}.whenever(requestService).isLastRequestWithKnownStatusDeletion(anyValueClass())
|
||||
|
||||
doAnswer {
|
||||
it.arguments[0] as String
|
||||
}.`when`(pseudonymizeService).patientPseudonym(any())
|
||||
}.whenever(pseudonymizeService).patientPseudonym(anyValueClass())
|
||||
|
||||
doAnswer {
|
||||
it.arguments[0]
|
||||
}.whenever(transformationService).transform(any())
|
||||
}.whenever(transformationService).transform(any<MtbFile>())
|
||||
|
||||
val mtbFile = MtbFile.builder()
|
||||
.withPatient(
|
||||
@ -206,32 +211,32 @@ class RequestProcessorTest {
|
||||
fun testShouldSendMtbFileAndSendSuccessEvent() {
|
||||
doAnswer {
|
||||
Request(
|
||||
id = 1L,
|
||||
uuid = UUID.randomUUID().toString(),
|
||||
patientId = "TEST_12345678901",
|
||||
pid = "P1",
|
||||
fingerprint = "different",
|
||||
type = RequestType.MTB_FILE,
|
||||
status = RequestStatus.SUCCESS,
|
||||
processedAt = Instant.parse("2023-08-08T02:00:00Z")
|
||||
1L,
|
||||
randomRequestId(),
|
||||
PatientPseudonym("TEST_12345678901"),
|
||||
PatientId("P1"),
|
||||
Fingerprint("different"),
|
||||
RequestType.MTB_FILE,
|
||||
RequestStatus.SUCCESS,
|
||||
Instant.parse("2023-08-08T02:00:00Z")
|
||||
)
|
||||
}.`when`(requestService).lastMtbFileRequestForPatientPseudonym(anyString())
|
||||
}.whenever(requestService).lastMtbFileRequestForPatientPseudonym(anyValueClass())
|
||||
|
||||
doAnswer {
|
||||
false
|
||||
}.`when`(requestService).isLastRequestWithKnownStatusDeletion(anyString())
|
||||
}.whenever(requestService).isLastRequestWithKnownStatusDeletion(anyValueClass())
|
||||
|
||||
doAnswer {
|
||||
MtbFileSender.Response(status = RequestStatus.SUCCESS)
|
||||
}.`when`(sender).send(any<MtbFileSender.MtbFileRequest>())
|
||||
}.whenever(sender).send(any<BwhcV1MtbFileRequest>())
|
||||
|
||||
doAnswer {
|
||||
it.arguments[0] as String
|
||||
}.`when`(pseudonymizeService).patientPseudonym(any())
|
||||
}.whenever(pseudonymizeService).patientPseudonym(anyValueClass())
|
||||
|
||||
doAnswer {
|
||||
it.arguments[0]
|
||||
}.whenever(transformationService).transform(any())
|
||||
}.whenever(transformationService).transform(any<MtbFile>())
|
||||
|
||||
val mtbFile = MtbFile.builder()
|
||||
.withPatient(
|
||||
@ -269,32 +274,32 @@ class RequestProcessorTest {
|
||||
fun testShouldSendMtbFileAndSendErrorEvent() {
|
||||
doAnswer {
|
||||
Request(
|
||||
id = 1L,
|
||||
uuid = UUID.randomUUID().toString(),
|
||||
patientId = "TEST_12345678901",
|
||||
pid = "P1",
|
||||
fingerprint = "different",
|
||||
type = RequestType.MTB_FILE,
|
||||
status = RequestStatus.SUCCESS,
|
||||
processedAt = Instant.parse("2023-08-08T02:00:00Z")
|
||||
1L,
|
||||
randomRequestId(),
|
||||
PatientPseudonym("TEST_12345678901"),
|
||||
PatientId("P1"),
|
||||
Fingerprint("different"),
|
||||
RequestType.MTB_FILE,
|
||||
RequestStatus.SUCCESS,
|
||||
Instant.parse("2023-08-08T02:00:00Z")
|
||||
)
|
||||
}.`when`(requestService).lastMtbFileRequestForPatientPseudonym(anyString())
|
||||
}.whenever(requestService).lastMtbFileRequestForPatientPseudonym(anyValueClass())
|
||||
|
||||
doAnswer {
|
||||
false
|
||||
}.`when`(requestService).isLastRequestWithKnownStatusDeletion(anyString())
|
||||
}.whenever(requestService).isLastRequestWithKnownStatusDeletion(anyValueClass())
|
||||
|
||||
doAnswer {
|
||||
MtbFileSender.Response(status = RequestStatus.ERROR)
|
||||
}.`when`(sender).send(any<MtbFileSender.MtbFileRequest>())
|
||||
}.whenever(sender).send(any<BwhcV1MtbFileRequest>())
|
||||
|
||||
doAnswer {
|
||||
it.arguments[0] as String
|
||||
}.`when`(pseudonymizeService).patientPseudonym(any())
|
||||
}.whenever(pseudonymizeService).patientPseudonym(anyValueClass())
|
||||
|
||||
doAnswer {
|
||||
it.arguments[0]
|
||||
}.whenever(transformationService).transform(any())
|
||||
}.whenever(transformationService).transform(any<MtbFile>())
|
||||
|
||||
val mtbFile = MtbFile.builder()
|
||||
.withPatient(
|
||||
@ -332,13 +337,13 @@ class RequestProcessorTest {
|
||||
fun testShouldSendDeleteRequestAndSaveUnknownRequestStatusAtFirst() {
|
||||
doAnswer {
|
||||
"PSEUDONYM"
|
||||
}.`when`(pseudonymizeService).patientPseudonym(anyString())
|
||||
}.whenever(pseudonymizeService).patientPseudonym(anyValueClass())
|
||||
|
||||
doAnswer {
|
||||
MtbFileSender.Response(status = RequestStatus.UNKNOWN)
|
||||
}.`when`(sender).send(any<MtbFileSender.DeleteRequest>())
|
||||
}.whenever(sender).send(any<DeleteRequest>())
|
||||
|
||||
this.requestProcessor.processDeletion("TEST_12345678901")
|
||||
this.requestProcessor.processDeletion(TEST_PATIENT_ID)
|
||||
|
||||
val requestCaptor = argumentCaptor<Request>()
|
||||
verify(requestService, times(1)).save(requestCaptor.capture())
|
||||
@ -350,13 +355,13 @@ class RequestProcessorTest {
|
||||
fun testShouldSendDeleteRequestAndSendSuccessEvent() {
|
||||
doAnswer {
|
||||
"PSEUDONYM"
|
||||
}.`when`(pseudonymizeService).patientPseudonym(anyString())
|
||||
}.whenever(pseudonymizeService).patientPseudonym(anyValueClass())
|
||||
|
||||
doAnswer {
|
||||
MtbFileSender.Response(status = RequestStatus.SUCCESS)
|
||||
}.`when`(sender).send(any<MtbFileSender.DeleteRequest>())
|
||||
}.whenever(sender).send(any<DeleteRequest>())
|
||||
|
||||
this.requestProcessor.processDeletion("TEST_12345678901")
|
||||
this.requestProcessor.processDeletion(TEST_PATIENT_ID)
|
||||
|
||||
val eventCaptor = argumentCaptor<ResponseEvent>()
|
||||
verify(applicationEventPublisher, times(1)).publishEvent(eventCaptor.capture())
|
||||
@ -368,13 +373,13 @@ class RequestProcessorTest {
|
||||
fun testShouldSendDeleteRequestAndSendErrorEvent() {
|
||||
doAnswer {
|
||||
"PSEUDONYM"
|
||||
}.`when`(pseudonymizeService).patientPseudonym(anyString())
|
||||
}.whenever(pseudonymizeService).patientPseudonym(anyValueClass())
|
||||
|
||||
doAnswer {
|
||||
MtbFileSender.Response(status = RequestStatus.ERROR)
|
||||
}.`when`(sender).send(any<MtbFileSender.DeleteRequest>())
|
||||
}.whenever(sender).send(any<DeleteRequest>())
|
||||
|
||||
this.requestProcessor.processDeletion("TEST_12345678901")
|
||||
this.requestProcessor.processDeletion(TEST_PATIENT_ID)
|
||||
|
||||
val eventCaptor = argumentCaptor<ResponseEvent>()
|
||||
verify(applicationEventPublisher, times(1)).publishEvent(eventCaptor.capture())
|
||||
@ -384,9 +389,9 @@ class RequestProcessorTest {
|
||||
|
||||
@Test
|
||||
fun testShouldSendDeleteRequestWithPseudonymErrorAndSaveErrorRequestStatus() {
|
||||
doThrow(RuntimeException()).`when`(pseudonymizeService).patientPseudonym(anyString())
|
||||
doThrow(RuntimeException()).whenever(pseudonymizeService).patientPseudonym(anyValueClass())
|
||||
|
||||
this.requestProcessor.processDeletion("TEST_12345678901")
|
||||
this.requestProcessor.processDeletion(TEST_PATIENT_ID)
|
||||
|
||||
val requestCaptor = argumentCaptor<Request>()
|
||||
verify(requestService, times(1)).save(requestCaptor.capture())
|
||||
@ -400,15 +405,15 @@ class RequestProcessorTest {
|
||||
|
||||
doAnswer {
|
||||
it.arguments[0] as String
|
||||
}.`when`(pseudonymizeService).patientPseudonym(any())
|
||||
}.whenever(pseudonymizeService).patientPseudonym(anyValueClass())
|
||||
|
||||
doAnswer {
|
||||
it.arguments[0]
|
||||
}.whenever(transformationService).transform(any())
|
||||
}.whenever(transformationService).transform(any<MtbFile>())
|
||||
|
||||
doAnswer {
|
||||
MtbFileSender.Response(status = RequestStatus.SUCCESS)
|
||||
}.`when`(sender).send(any<MtbFileSender.MtbFileRequest>())
|
||||
}.whenever(sender).send(any<BwhcV1MtbFileRequest>())
|
||||
|
||||
val mtbFile = MtbFile.builder()
|
||||
.withPatient(
|
||||
@ -442,4 +447,8 @@ class RequestProcessorTest {
|
||||
assertThat(eventCaptor.firstValue.status).isEqualTo(RequestStatus.SUCCESS)
|
||||
}
|
||||
|
||||
companion object {
|
||||
val TEST_PATIENT_ID = PatientId("TEST_12345678901")
|
||||
}
|
||||
|
||||
}
|
@ -19,6 +19,7 @@
|
||||
|
||||
package dev.dnpm.etl.processor.services
|
||||
|
||||
import dev.dnpm.etl.processor.*
|
||||
import dev.dnpm.etl.processor.monitoring.Request
|
||||
import dev.dnpm.etl.processor.monitoring.RequestRepository
|
||||
import dev.dnpm.etl.processor.monitoring.RequestStatus
|
||||
@ -30,8 +31,9 @@ import org.junit.jupiter.api.extension.ExtendWith
|
||||
import org.mockito.Mock
|
||||
import org.mockito.Mockito.*
|
||||
import org.mockito.junit.jupiter.MockitoExtension
|
||||
import org.mockito.kotlin.anyValueClass
|
||||
import org.mockito.kotlin.whenever
|
||||
import java.time.Instant
|
||||
import java.util.*
|
||||
|
||||
@ExtendWith(MockitoExtension::class)
|
||||
class RequestServiceTest {
|
||||
@ -41,14 +43,14 @@ class RequestServiceTest {
|
||||
private lateinit var requestService: RequestService
|
||||
|
||||
private fun anyRequest() = any(Request::class.java) ?: Request(
|
||||
id = 0L,
|
||||
uuid = UUID.randomUUID().toString(),
|
||||
patientId = "TEST_dummy",
|
||||
pid = "PX",
|
||||
fingerprint = "dummy",
|
||||
type = RequestType.MTB_FILE,
|
||||
status = RequestStatus.SUCCESS,
|
||||
processedAt = Instant.parse("2023-08-08T02:00:00Z")
|
||||
0L,
|
||||
randomRequestId(),
|
||||
PatientPseudonym("TEST_dummy"),
|
||||
PatientId("PX"),
|
||||
Fingerprint("dummy"),
|
||||
RequestType.MTB_FILE,
|
||||
RequestStatus.SUCCESS,
|
||||
Instant.parse("2023-08-08T02:00:00Z")
|
||||
)
|
||||
|
||||
@BeforeEach
|
||||
@ -63,34 +65,34 @@ class RequestServiceTest {
|
||||
fun shouldIndicateLastRequestIsDeleteRequest() {
|
||||
val requests = listOf(
|
||||
Request(
|
||||
id = 1L,
|
||||
uuid = UUID.randomUUID().toString(),
|
||||
patientId = "TEST_12345678901",
|
||||
pid = "P1",
|
||||
fingerprint = "0123456789abcdef1",
|
||||
type = RequestType.MTB_FILE,
|
||||
status = RequestStatus.WARNING,
|
||||
processedAt = Instant.parse("2023-07-07T00:00:00Z")
|
||||
1L,
|
||||
randomRequestId(),
|
||||
PatientPseudonym("TEST_12345678901"),
|
||||
PatientId("P1"),
|
||||
Fingerprint("0123456789abcdef1"),
|
||||
RequestType.MTB_FILE,
|
||||
RequestStatus.WARNING,
|
||||
Instant.parse("2023-07-07T00:00:00Z")
|
||||
),
|
||||
Request(
|
||||
id = 2L,
|
||||
uuid = UUID.randomUUID().toString(),
|
||||
patientId = "TEST_12345678901",
|
||||
pid = "P1",
|
||||
fingerprint = "0123456789abcdefd",
|
||||
type = RequestType.DELETE,
|
||||
status = RequestStatus.WARNING,
|
||||
processedAt = Instant.parse("2023-07-07T02:00:00Z")
|
||||
2L,
|
||||
randomRequestId(),
|
||||
PatientPseudonym("TEST_12345678901"),
|
||||
PatientId("P1"),
|
||||
Fingerprint("0123456789abcdefd"),
|
||||
RequestType.DELETE,
|
||||
RequestStatus.WARNING,
|
||||
Instant.parse("2023-07-07T02:00:00Z")
|
||||
),
|
||||
Request(
|
||||
id = 3L,
|
||||
uuid = UUID.randomUUID().toString(),
|
||||
patientId = "TEST_12345678901",
|
||||
pid = "P1",
|
||||
fingerprint = "0123456789abcdef1",
|
||||
type = RequestType.MTB_FILE,
|
||||
status = RequestStatus.UNKNOWN,
|
||||
processedAt = Instant.parse("2023-08-11T00:00:00Z")
|
||||
3L,
|
||||
randomRequestId(),
|
||||
PatientPseudonym("TEST_12345678901"),
|
||||
PatientId("P1"),
|
||||
Fingerprint("0123456789abcdef1"),
|
||||
RequestType.MTB_FILE,
|
||||
RequestStatus.UNKNOWN,
|
||||
Instant.parse("2023-08-11T00:00:00Z")
|
||||
)
|
||||
)
|
||||
|
||||
@ -103,34 +105,34 @@ class RequestServiceTest {
|
||||
fun shouldIndicateLastRequestIsNotDeleteRequest() {
|
||||
val requests = listOf(
|
||||
Request(
|
||||
id = 1L,
|
||||
uuid = UUID.randomUUID().toString(),
|
||||
patientId = "TEST_12345678901",
|
||||
pid = "P1",
|
||||
fingerprint = "0123456789abcdef1",
|
||||
type = RequestType.MTB_FILE,
|
||||
status = RequestStatus.WARNING,
|
||||
processedAt = Instant.parse("2023-07-07T00:00:00Z")
|
||||
1L,
|
||||
randomRequestId(),
|
||||
PatientPseudonym("TEST_12345678901"),
|
||||
PatientId("P1"),
|
||||
Fingerprint("0123456789abcdef1"),
|
||||
RequestType.MTB_FILE,
|
||||
RequestStatus.WARNING,
|
||||
Instant.parse("2023-07-07T00:00:00Z")
|
||||
),
|
||||
Request(
|
||||
id = 2L,
|
||||
uuid = UUID.randomUUID().toString(),
|
||||
patientId = "TEST_12345678901",
|
||||
pid = "P1",
|
||||
fingerprint = "0123456789abcdef1",
|
||||
type = RequestType.MTB_FILE,
|
||||
status = RequestStatus.WARNING,
|
||||
processedAt = Instant.parse("2023-07-07T02:00:00Z")
|
||||
2L,
|
||||
randomRequestId(),
|
||||
PatientPseudonym("TEST_12345678901"),
|
||||
PatientId("P1"),
|
||||
Fingerprint("0123456789abcdef1"),
|
||||
RequestType.MTB_FILE,
|
||||
RequestStatus.WARNING,
|
||||
Instant.parse("2023-07-07T02:00:00Z")
|
||||
),
|
||||
Request(
|
||||
id = 3L,
|
||||
uuid = UUID.randomUUID().toString(),
|
||||
patientId = "TEST_12345678901",
|
||||
pid = "P1",
|
||||
fingerprint = "0123456789abcdef1",
|
||||
type = RequestType.MTB_FILE,
|
||||
status = RequestStatus.UNKNOWN,
|
||||
processedAt = Instant.parse("2023-08-11T00:00:00Z")
|
||||
3L,
|
||||
randomRequestId(),
|
||||
PatientPseudonym("TEST_12345678901"),
|
||||
PatientId("P1"),
|
||||
Fingerprint("0123456789abcdef1"),
|
||||
RequestType.MTB_FILE,
|
||||
RequestStatus.UNKNOWN,
|
||||
Instant.parse("2023-08-11T00:00:00Z")
|
||||
)
|
||||
)
|
||||
|
||||
@ -143,31 +145,31 @@ class RequestServiceTest {
|
||||
fun shouldReturnPatientsLastRequest() {
|
||||
val requests = listOf(
|
||||
Request(
|
||||
id = 1L,
|
||||
uuid = UUID.randomUUID().toString(),
|
||||
patientId = "TEST_12345678901",
|
||||
pid = "P1",
|
||||
fingerprint = "0123456789abcdef1",
|
||||
type = RequestType.DELETE,
|
||||
status = RequestStatus.SUCCESS,
|
||||
processedAt = Instant.parse("2023-07-07T02:00:00Z")
|
||||
1L,
|
||||
randomRequestId(),
|
||||
PatientPseudonym("TEST_12345678901"),
|
||||
PatientId("P1"),
|
||||
Fingerprint("0123456789abcdef1"),
|
||||
RequestType.DELETE,
|
||||
RequestStatus.SUCCESS,
|
||||
Instant.parse("2023-07-07T02:00:00Z")
|
||||
),
|
||||
Request(
|
||||
id = 1L,
|
||||
uuid = UUID.randomUUID().toString(),
|
||||
patientId = "TEST_12345678902",
|
||||
pid = "P2",
|
||||
fingerprint = "0123456789abcdef2",
|
||||
type = RequestType.MTB_FILE,
|
||||
status = RequestStatus.WARNING,
|
||||
processedAt = Instant.parse("2023-08-08T00:00:00Z")
|
||||
1L,
|
||||
randomRequestId(),
|
||||
PatientPseudonym("TEST_12345678902"),
|
||||
PatientId("P2"),
|
||||
Fingerprint("0123456789abcdef2"),
|
||||
RequestType.MTB_FILE,
|
||||
RequestStatus.WARNING,
|
||||
Instant.parse("2023-08-08T00:00:00Z")
|
||||
)
|
||||
)
|
||||
|
||||
val actual = RequestService.lastMtbFileRequestForPatientPseudonym(requests)
|
||||
|
||||
assertThat(actual).isInstanceOf(Request::class.java)
|
||||
assertThat(actual?.fingerprint).isEqualTo("0123456789abcdef2")
|
||||
assertThat(actual?.fingerprint).isEqualTo(Fingerprint("0123456789abcdef2"))
|
||||
}
|
||||
|
||||
@Test
|
||||
@ -184,16 +186,16 @@ class RequestServiceTest {
|
||||
doAnswer {
|
||||
val obj = it.arguments[0] as Request
|
||||
obj.copy(id = 1L)
|
||||
}.`when`(requestRepository).save(anyRequest())
|
||||
}.whenever(requestRepository).save(anyRequest())
|
||||
|
||||
val request = Request(
|
||||
uuid = UUID.randomUUID().toString(),
|
||||
patientId = "TEST_12345678901",
|
||||
pid = "P1",
|
||||
fingerprint = "0123456789abcdef1",
|
||||
type = RequestType.DELETE,
|
||||
status = RequestStatus.SUCCESS,
|
||||
processedAt = Instant.parse("2023-07-07T02:00:00Z")
|
||||
randomRequestId(),
|
||||
PatientPseudonym("TEST_12345678901"),
|
||||
PatientId("P1"),
|
||||
Fingerprint("0123456789abcdef1"),
|
||||
RequestType.DELETE,
|
||||
RequestStatus.SUCCESS,
|
||||
Instant.parse("2023-07-07T02:00:00Z")
|
||||
)
|
||||
|
||||
requestService.save(request)
|
||||
@ -203,23 +205,23 @@ class RequestServiceTest {
|
||||
|
||||
@Test
|
||||
fun allRequestsByPatientPseudonymShouldRequestAllRequestsForPatientPseudonym() {
|
||||
requestService.allRequestsByPatientPseudonym("TEST_12345678901")
|
||||
requestService.allRequestsByPatientPseudonym(PatientPseudonym("TEST_12345678901"))
|
||||
|
||||
verify(requestRepository, times(1)).findAllByPatientIdOrderByProcessedAtDesc(anyString())
|
||||
verify(requestRepository, times(1)).findAllByPatientPseudonymOrderByProcessedAtDesc(anyValueClass())
|
||||
}
|
||||
|
||||
@Test
|
||||
fun lastMtbFileRequestForPatientPseudonymShouldRequestAllRequestsForPatientPseudonym() {
|
||||
requestService.lastMtbFileRequestForPatientPseudonym("TEST_12345678901")
|
||||
requestService.lastMtbFileRequestForPatientPseudonym(PatientPseudonym("TEST_12345678901"))
|
||||
|
||||
verify(requestRepository, times(1)).findAllByPatientIdOrderByProcessedAtDesc(anyString())
|
||||
verify(requestRepository, times(1)).findAllByPatientPseudonymOrderByProcessedAtDesc(anyValueClass())
|
||||
}
|
||||
|
||||
@Test
|
||||
fun isLastRequestDeletionShouldRequestAllRequestsForPatientPseudonym() {
|
||||
requestService.isLastRequestWithKnownStatusDeletion("TEST_12345678901")
|
||||
requestService.isLastRequestWithKnownStatusDeletion(PatientPseudonym("TEST_12345678901"))
|
||||
|
||||
verify(requestRepository, times(1)).findAllByPatientIdOrderByProcessedAtDesc(anyString())
|
||||
verify(requestRepository, times(1)).findAllByPatientPseudonymOrderByProcessedAtDesc(anyValueClass())
|
||||
}
|
||||
|
||||
}
|
@ -19,8 +19,8 @@
|
||||
|
||||
package dev.dnpm.etl.processor.services
|
||||
|
||||
import dev.dnpm.etl.processor.*
|
||||
import dev.dnpm.etl.processor.monitoring.Request
|
||||
import dev.dnpm.etl.processor.monitoring.RequestRepository
|
||||
import dev.dnpm.etl.processor.monitoring.RequestStatus
|
||||
import dev.dnpm.etl.processor.monitoring.RequestType
|
||||
import org.assertj.core.api.Assertions.assertThat
|
||||
@ -29,7 +29,6 @@ import org.junit.jupiter.api.Test
|
||||
import org.junit.jupiter.api.extension.ExtendWith
|
||||
import org.junit.jupiter.params.ParameterizedTest
|
||||
import org.junit.jupiter.params.provider.MethodSource
|
||||
import org.mockito.ArgumentMatchers.anyString
|
||||
import org.mockito.Mock
|
||||
import org.mockito.junit.jupiter.MockitoExtension
|
||||
import org.mockito.kotlin.*
|
||||
@ -40,64 +39,64 @@ import java.util.*
|
||||
@ExtendWith(MockitoExtension::class)
|
||||
class ResponseProcessorTest {
|
||||
|
||||
private lateinit var requestRepository: RequestRepository
|
||||
private lateinit var requestService: RequestService
|
||||
private lateinit var statisticsUpdateProducer: Sinks.Many<Any>
|
||||
|
||||
private lateinit var responseProcessor: ResponseProcessor
|
||||
|
||||
private val testRequest = Request(
|
||||
1L,
|
||||
"TestID1234",
|
||||
"PSEUDONYM-A",
|
||||
"1",
|
||||
"dummyfingerprint",
|
||||
RequestId("TestID1234"),
|
||||
PatientPseudonym("PSEUDONYM-A"),
|
||||
PatientId("1"),
|
||||
Fingerprint("dummyfingerprint"),
|
||||
RequestType.MTB_FILE,
|
||||
RequestStatus.UNKNOWN
|
||||
)
|
||||
|
||||
@BeforeEach
|
||||
fun setup(
|
||||
@Mock requestRepository: RequestRepository,
|
||||
@Mock requestService: RequestService,
|
||||
@Mock statisticsUpdateProducer: Sinks.Many<Any>
|
||||
) {
|
||||
this.requestRepository = requestRepository
|
||||
this.requestService = requestService
|
||||
this.statisticsUpdateProducer = statisticsUpdateProducer
|
||||
|
||||
this.responseProcessor = ResponseProcessor(requestRepository, statisticsUpdateProducer)
|
||||
this.responseProcessor = ResponseProcessor(requestService, statisticsUpdateProducer)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun shouldNotSaveStatusForUnknownRequest() {
|
||||
doAnswer {
|
||||
Optional.empty<Request>()
|
||||
}.whenever(requestRepository).findByUuidEquals(anyString())
|
||||
}.whenever(requestService).findByUuid(anyValueClass())
|
||||
|
||||
val event = ResponseEvent(
|
||||
"TestID1234",
|
||||
RequestId("TestID1234"),
|
||||
Instant.parse("2023-09-09T00:00:00Z"),
|
||||
RequestStatus.SUCCESS
|
||||
)
|
||||
|
||||
this.responseProcessor.handleResponseEvent(event)
|
||||
|
||||
verify(requestRepository, never()).save(any())
|
||||
verify(requestService, never()).save(any())
|
||||
}
|
||||
|
||||
@Test
|
||||
fun shouldNotSaveStatusWithUnknownState() {
|
||||
doAnswer {
|
||||
Optional.of(testRequest)
|
||||
}.whenever(requestRepository).findByUuidEquals(anyString())
|
||||
}.whenever(requestService).findByUuid(anyValueClass())
|
||||
|
||||
val event = ResponseEvent(
|
||||
"TestID1234",
|
||||
RequestId("TestID1234"),
|
||||
Instant.parse("2023-09-09T00:00:00Z"),
|
||||
RequestStatus.UNKNOWN
|
||||
)
|
||||
|
||||
this.responseProcessor.handleResponseEvent(event)
|
||||
|
||||
verify(requestRepository, never()).save(any())
|
||||
verify(requestService, never()).save(any<Request>())
|
||||
}
|
||||
|
||||
@ParameterizedTest
|
||||
@ -105,10 +104,10 @@ class ResponseProcessorTest {
|
||||
fun shouldSaveStatusForKnownRequest(requestStatus: RequestStatus) {
|
||||
doAnswer {
|
||||
Optional.of(testRequest)
|
||||
}.whenever(requestRepository).findByUuidEquals(anyString())
|
||||
}.whenever(requestService).findByUuid(anyValueClass())
|
||||
|
||||
val event = ResponseEvent(
|
||||
"TestID1234",
|
||||
RequestId("TestID1234"),
|
||||
Instant.parse("2023-09-09T00:00:00Z"),
|
||||
requestStatus
|
||||
)
|
||||
@ -116,7 +115,7 @@ class ResponseProcessorTest {
|
||||
this.responseProcessor.handleResponseEvent(event)
|
||||
|
||||
val captor = argumentCaptor<Request>()
|
||||
verify(requestRepository, times(1)).save(captor.capture())
|
||||
verify(requestService, times(1)).save(captor.capture())
|
||||
assertThat(captor.firstValue).isNotNull
|
||||
assertThat(captor.firstValue.status).isEqualTo(requestStatus)
|
||||
}
|
||||
|
File diff suppressed because one or more lines are too long
2243
src/test/resources/mv64e-mtb-fake-patient.json
Normal file
2243
src/test/resources/mv64e-mtb-fake-patient.json
Normal file
File diff suppressed because it is too large
Load Diff
Loading…
x
Reference in New Issue
Block a user