mirror of
https://github.com/pcvolkmer/etl-processor.git
synced 2025-07-01 06:02:54 +00:00
Compare commits
52 Commits
Author | SHA1 | Date | |
---|---|---|---|
a31d2b4bcc | |||
67d5fb4c67 | |||
329be65d1a | |||
91fe3d1c23 | |||
f4b86ce2ea | |||
19d0daa442 | |||
cc9811d11d | |||
8ce5b06823 | |||
3cc34fb30b | |||
17e04a3f89 | |||
f71a775e12 | |||
45c83e943b | |||
6dcbfde62e | |||
4cdc419478 | |||
90b529adb4 | |||
a3bc60986b | |||
f5df0b5d22 | |||
972ac745e9 | |||
358373cf70 | |||
27a62321fa | |||
30cf0fd22e | |||
531a8589db | |||
fa89a64ddd | |||
45ad5e8827 | |||
c4eb4d0fe2 | |||
4bc69a353c | |||
9d30f750f7 | |||
a1a252d5a9 | |||
568942bfe5 | |||
15f0432553 | |||
113bf2dd2e | |||
7ac151202a | |||
5d9d47c2df | |||
585468314c | |||
441bff3783 | |||
21959c1698 | |||
8a11e6e85b | |||
5579ad1453 | |||
c2026bdd07 | |||
de6faecb02 | |||
3be8bc53ff | |||
fad2f33fd6 | |||
d88e2973da | |||
af767e4ea6 | |||
f98c970348 | |||
75872a149f | |||
e24ba430a5 | |||
08914a6f86 | |||
104f50afcb | |||
0083e75940 | |||
c892ff2461 | |||
4a9cffbaa5 |
2
.gitignore
vendored
2
.gitignore
vendored
@ -5,6 +5,8 @@ build/
|
||||
!**/src/main/**/build/
|
||||
!**/src/test/**/build/
|
||||
|
||||
bindings/ca-certificates/*.pem
|
||||
|
||||
### STS ###
|
||||
.apt_generated
|
||||
.classpath
|
||||
|
193
README.md
193
README.md
@ -2,29 +2,33 @@
|
||||
|
||||
Diese Anwendung versendet ein bwHC-MTB-File an das bwHC-Backend und pseudonymisiert die Patienten-ID.
|
||||
|
||||
### Einordnung innerhalb einer DNPM-ETL-Strecke
|
||||
## Einordnung innerhalb einer DNPM-ETL-Strecke
|
||||
|
||||
Diese Anwendung erlaubt das Entgegennehmen HTTP/REST-Anfragen aus dem Onkostar-Plugin **[onkostar-plugin-dnpmexport](https://github.com/CCC-MF/onkostar-plugin-dnpmexport)**.
|
||||
Diese Anwendung erlaubt das Entgegennehmen von HTTP/REST-Anfragen aus dem Onkostar-Plugin **[onkostar-plugin-dnpmexport](https://github.com/CCC-MF/onkostar-plugin-dnpmexport)**.
|
||||
|
||||
Der Inhalt einer Anfrage, wenn ein bwHC-MTBFile, wird pseudonymisiert und auf Duplikate geprüft.
|
||||
Duplikate werden verworfen, Änderungen werden weitergeleitet.
|
||||
|
||||
Löschanfragen werden immer als Löschanfrage an das bwHC-backend weitergeleitet.
|
||||
|
||||
Zudem ist eine minimalistische Weboberfläche integriert, die einen Einblick in den aktuellen Zustand der Anwendung gewährt.
|
||||
|
||||

|
||||
|
||||
#### HTTP/REST-Konfiguration
|
||||
### Datenübermittlung über HTTP/REST
|
||||
|
||||
Anfragen werden, wenn nicht als Duplikat behandelt, nach der Pseudonymisierung direkt an das bwHC-Backend gesendet.
|
||||
|
||||
#### Konfiguration für Apache Kafka
|
||||
### Datenübermittlung mit Apache Kafka
|
||||
|
||||
Anfragen werden, wenn nicht als Duplikat behandelt, nach der Pseudonymisierung an Apache Kafka übergeben.
|
||||
Eine Antwort wird dabei ebenfalls mithilfe von Apache Kafka übermittelt und nach der Entgegennahme verarbeitet.
|
||||
|
||||
Siehe hierzu auch: https://github.com/CCC-MF/kafka-to-bwhc
|
||||
|
||||
## Pseudonymisierung der Patienten-ID
|
||||
## Konfiguration
|
||||
|
||||
### Pseudonymisierung der Patienten-ID
|
||||
|
||||
Wenn eine URI zu einer gPAS-Instanz (Version >= 2023.1.0) angegeben ist, wird diese verwendet.
|
||||
Ist diese nicht gesetzt. wird intern eine Anonymisierung der Patienten-ID vorgenommen.
|
||||
@ -32,24 +36,104 @@ Ist diese nicht gesetzt. wird intern eine Anonymisierung der Patienten-ID vorgen
|
||||
* `APP_PSEUDONYMIZE_PREFIX`: Standortbezogenes Prefix - `UNKNOWN`, wenn nicht gesetzt
|
||||
* `APP_PSEUDONYMIZE_GENERATOR`: `BUILDIN` oder `GPAS` - `BUILDIN`, wenn nicht gesetzt
|
||||
|
||||
### Eingebaute Pseudonymisierung
|
||||
**Hinweise**:
|
||||
|
||||
Wurde keine oder die Verwendung der eingebauten Pseudonymisierung konfiguriert, so wird für die Patienten-ID der
|
||||
* Der alte Konfigurationsparameter `APP_PSEUDONYMIZER` mit den Werten `GPAS` oder `BUILDIN` sollte nicht
|
||||
mehr verwendet werden.
|
||||
* Die Pseudonymisierung erfolgt im ETL-Prozessor nur für die Patienten-ID.
|
||||
Andere Referenz-IDs werden nicht anonymisiert.
|
||||
Dies erfolgt bei Nutzung von **[onkostar-plugin-dnpmexport](https://github.com/CCC-MF/onkostar-plugin-dnpmexport)**
|
||||
bereits im Plugin selbst.
|
||||
|
||||
#### Eingebaute Anonymisierung
|
||||
|
||||
Wurde keine oder die Verwendung der eingebauten Anonymisierung konfiguriert, so wird für die Patienten-ID der
|
||||
entsprechende SHA-256-Hash gebildet und Base64-codiert - hier ohne endende "=" - zuzüglich des konfigurierten Prefixes
|
||||
als Patienten-Pseudonym verwendet.
|
||||
|
||||
### Pseudonymisierung mit gPAS
|
||||
#### Pseudonymisierung mit gPAS
|
||||
|
||||
Wurde die Verwendung von gPAS konfiguriert, so sind weitere Angaben zu konfigurieren.
|
||||
|
||||
* `APP_PSEUDONYMIZE_GPAS_URI`: URI der gPAS-Instanz inklusive Endpoint (
|
||||
z.B. `http://localhost:8080/ttp-fhir/fhir/gpas/$$pseudonymizeAllowCreate`)
|
||||
* `APP_PSEUDONYMIZE_GPAS_URI`: URI der gPAS-Instanz inklusive Endpoint (z.B. `http://localhost:8080/ttp-fhir/fhir/gpas/$$pseudonymizeAllowCreate`)
|
||||
* `APP_PSEUDONYMIZE_GPAS_TARGET`: gPas Domänenname
|
||||
* `APP_PSEUDONYMIZE_GPAS_USERNAME`: gPas Basic-Auth Benutzername
|
||||
* `APP_PSEUDONYMIZE_GPAS_PASSWORD`: gPas Basic-Auth Passwort
|
||||
* `APP_PSEUDONYMIZE_GPAS_SSLCALOCATION`: Root Zertifikat für gPas, falls es dediziert hinzugefügt werden muss.
|
||||
|
||||
## Transformation von Werten
|
||||
### Anmeldung mit einem Passwort
|
||||
|
||||
Ein initialer Administrator-Account kann optional konfiguriert werden und sorgt dafür, dass bestimmte Bereiche nur nach
|
||||
einem erfolgreichen Login erreichbar sind.
|
||||
|
||||
* `APP_SECURITY_ADMIN_USER`: Muss angegeben werden zur Aktivierung der Zugriffsbeschränkung.
|
||||
* `APP_SECURITY_ADMIN_PASSWORD`: Das Passwort für den Administrator (Empfohlen).
|
||||
|
||||
Ein Administrator-Passwort muss inklusive des Encoding-Prefixes vorliegen.
|
||||
|
||||
Hier Beispiele für das Beispielpasswort `very-secret`:
|
||||
|
||||
* `{noop}very-secret` (Das Passwort liegt im Klartext vor - nicht empfohlen!)
|
||||
* `{bcrypt}$2y$05$CCkfsMr/wbTleMyjVIK8g.Aa3RCvrvoLXVAsL.f6KeouS88vXD9b6`
|
||||
* `{sha256}9a34717f0646b5e9cfcba70055de62edb026ff4f68671ba3db96aa29297d2df5f1a037d58c745657`
|
||||
|
||||
Wird kein Administrator-Passwort angegeben, wird ein zufälliger Wert generiert und beim Start der Anwendung in den Logs
|
||||
angezeigt.
|
||||
|
||||
#### Weitere (nicht administrative) Nutzer mit OpenID Connect
|
||||
|
||||
Die folgenden Konfigurationsparameter werden benötigt, um die Authentifizierung weiterer Benutzer an einen OIDC-Provider
|
||||
zu delegieren.
|
||||
Ein Admin-Benutzer muss dabei konfiguriert sein.
|
||||
|
||||
* `APP_SECURITY_ENABLE_OIDC`: Aktiviert die Nutzung von OpenID Connect. Damit sind weitere Parameter erforderlich
|
||||
* `SPRING_SECURITY_OAUTH2_CLIENT_REGISTRATION_CUSTOM_CLIENT_NAME`: Name. Wird beim zusätzlichen Loginbutton angezeigt.
|
||||
* `SPRING_SECURITY_OAUTH2_CLIENT_REGISTRATION_CUSTOM_CLIENT_ID`: Client-ID
|
||||
* `SPRING_SECURITY_OAUTH2_CLIENT_REGISTRATION_CUSTOM_CLIENT_SECRET`: Client-Secret
|
||||
* `SPRING_SECURITY_OAUTH2_CLIENT_REGISTRATION_CUSTOM_CLIENT_SCOPE[0]`: Hier sollte immer `openid` angegeben werden.
|
||||
* `SPRING_SECURITY_OAUTH2_CLIENT_PROVIDER_CUSTOM_ISSUER_URI`: Die URI des Providers,
|
||||
z.B. `https://auth.example.com/realm/example`
|
||||
* `SPRING_SECURITY_OAUTH2_CLIENT_PROVIDER_CUSTOM_USER_NAME_ATTRIBUTE`: Name des Attributes, welches den Benutzernamen
|
||||
enthält.
|
||||
Oft verwendet: `preferred_username`
|
||||
|
||||
Ist die Nutzung von OpenID Connect konfiguriert, erscheint ein zusätzlicher Login-Button zur Nutzung mit OpenID Connect
|
||||
und dem konfigurierten `CLIENT_NAME`.
|
||||
|
||||

|
||||
|
||||
Weitere Informationen zur Konfiguration des OIDC-Providers
|
||||
sind [hier](https://docs.spring.io/spring-security/reference/servlet/oauth2/index.html#oauth2-client)
|
||||
zu finden.
|
||||
|
||||
#### Auswirkungen auf den dargestellten Inhalt
|
||||
|
||||
Nur Administratoren haben Zugriff auf den Konfigurationsbereich, nur angemeldete Benutzer können die anonymisierte oder
|
||||
pseudonymisierte Patienten-ID sowie den Qualitätsbericht des bwHC-Backends einsehen.
|
||||
|
||||
Wurde kein Administrator-Account konfiguriert, sind diese Inhalte generell nicht verfügbar.
|
||||
|
||||
### Tokenbasierte Authentifizierung für MTBFile-Endpunkt
|
||||
|
||||
Die Anwendung unterstützt das Erstellen und Nutzen einer tokenbasierten Authentifizierung für den MTB-File-Endpunkt.
|
||||
|
||||
Dies kann mit der Umgebungsvariable `APP_SECURITY_ENABLE_TOKENS` aktiviert (`true` oder `false`) werden
|
||||
und ist als Standardeinstellung nicht aktiv.
|
||||
|
||||
Ist diese Einstellung aktiviert worden, ist es Administratoren möglich, Zugriffstokens für Onkostar zu erstellen, die
|
||||
zur Nutzung des MTB-File-Endpunkts eine HTTP-Basic-Authentifizierung voraussetzen.
|
||||
|
||||

|
||||
|
||||
In diesem Fall können den Endpunkt für das Onkostar-Plugin **[onkostar-plugin-dnpmexport](https://github.com/CCC-MF/onkostar-plugin-dnpmexport)** wie folgt konfigurieren:
|
||||
|
||||
```
|
||||
https://testonkostar:MTg1NTL...NGU4@etl.example.com/mtbfile
|
||||
```
|
||||
|
||||
Ist die Verwendung von Tokens aktiv, werden Anfragen ohne die Angabe der Token-Information abgelehnt.
|
||||
|
||||
### Transformation von Werten
|
||||
|
||||
In Onkostar kann es vorkommen, dass ein Wert eines Merkmalskatalogs an einem Standort angepasst wurde und dadurch nicht dem Wert entspricht,
|
||||
der vom bwHC-Backend akzeptiert wird.
|
||||
@ -57,26 +141,26 @@ der vom bwHC-Backend akzeptiert wird.
|
||||
Diese Anwendung bietet daher die Möglichkeit, eine Transformation vorzunehmen. Hierzu muss der "Pfad" innerhalb des JSON-MTB-Files angegeben werden und
|
||||
welcher Wert wie ersetzt werden soll.
|
||||
|
||||
Hier ein Beispiel für die erste (Index 0 - weitere dann mit 1,2,...) Transformationsregel:
|
||||
Hier ein Beispiel für die erste (Index 0 - weitere dann mit 1,2, ...) Transformationsregel:
|
||||
|
||||
* `APP_TRANSFORMATIONS_0_PATH`: Pfad zum Wert in der JSON-MTB-Datei. Beispiel: `diagnoses[*].icd10.version` für **alle** Diagnosen
|
||||
* `APP_TRANSFORMATIONS_0_FROM`: Angabe des Werts, der ersetzt werden soll. Andere Werte bleiben dabei unverändert.
|
||||
* `APP_TRANSFORMATIONS_0_TO`: Angabe des neuen Werts.
|
||||
|
||||
## Mögliche Endpunkte
|
||||
### Mögliche Endpunkte zur Datenübermittlung
|
||||
|
||||
Für REST-Requests als auch zur Nutzung von Kafka-Topics können Endpunkte konfiguriert werden.
|
||||
|
||||
Es ist dabei nur die Konfiguration eines Endpunkts zulässig.
|
||||
Werden sowohl REST als auch Kafka-Endpunkt konfiguriert, wird nur der REST-Endpunkt verwendet.
|
||||
|
||||
### REST
|
||||
#### REST
|
||||
|
||||
Folgende Umgebungsvariablen müssen gesetzt sein, damit ein bwHC-MTB-File an das bwHC-Backend gesendet wird:
|
||||
|
||||
* `APP_REST_URI`: URI der zu benutzenden API der bwHC-Backend-Instanz. z.B.: `http://localhost:9000/bwhc/etl/api`
|
||||
|
||||
### Kafka-Topics
|
||||
#### Kafka-Topics
|
||||
|
||||
Folgende Umgebungsvariablen müssen gesetzt sein, damit ein bwHC-MTB-File an ein Kafka-Topic übermittelt wird:
|
||||
|
||||
@ -92,7 +176,7 @@ Weitere Einstellungen können über die Parameter von Spring Kafka konfiguriert
|
||||
Lässt sich keine Verbindung zu dem bwHC-Backend aufbauen, wird eine Rückantwort mit Status-Code `900` erwartet, welchen es
|
||||
für HTTP nicht gibt.
|
||||
|
||||
#### Retention Time
|
||||
##### Retention Time
|
||||
|
||||
Generell werden in Apache Kafka alle Records entsprechend der Konfiguration vorgehalten.
|
||||
So wird ohne spezielle Konfiguration ein Record für 7 Tage in Apache Kafka gespeichert.
|
||||
@ -105,7 +189,7 @@ Beispiel - auszuführen innerhalb des Kafka-Containers: Löschen alter Records n
|
||||
kafka-configs.sh --bootstrap-server localhost:9092 --alter --topic test --add-config retention.ms=86400000
|
||||
```
|
||||
|
||||
#### Key based Retention
|
||||
##### Key based Retention
|
||||
|
||||
Möchten Sie hingegen immer nur die letzte Meldung für einen Patienten und eine Erkrankung in Apache Kafka vorhalten,
|
||||
so ist die nachfolgend genannte Konfiguration der Kafka-Topics hilfreich.
|
||||
@ -141,8 +225,37 @@ Diese Anwendung ist auch als Docker-Image verfügbar: https://github.com/CCC-MF/
|
||||
./gradlew bootBuildImage
|
||||
```
|
||||
|
||||
### Integration eines eigenen Root CA Zertifikats
|
||||
|
||||
Wird eine eigene Root CA verwendet, die nicht offiziell signiert ist, wird es zu Problemen beim SSL-Handshake kommen, wenn z.B. gPAS zur Generierung von Pseudonymen verwendet wird.
|
||||
|
||||
Hier bietet es sich an, das Root CA Zertifikat in das Image zu integrieren.
|
||||
|
||||
#### Integration beim Bauen des Images
|
||||
|
||||
Hier muss die Zeile `"BP_EMBED_CERTS" to "true"` in der Datei `build.gradle.kts` verwendet werden und darf nicht als Kommentar verwendet werden.
|
||||
|
||||
Die PEM-Datei mit dem/den Root CA Zertifikat(en) muss dabei im vorbereiteten Verzeichnis [`bindings/ca-certificates`](bindings/ca-certificates) enthalten sein.
|
||||
|
||||
#### Integration zur Laufzeit
|
||||
|
||||
Hier muss die Umgebungsvariable `SERVICE_BINDING_ROOT` z.B. auf den Wert `/bindings` gesetzt sein.
|
||||
Zudem muss ein Verzeichnis `bindings/ca-certificates` - analog zum Verzeichnis [`bindings/ca-certificates`](bindings/ca-certificates) mit einer PEM-Datei als Docker-Volume eingebunden werden.
|
||||
|
||||
Beispiel für Docker-Compose:
|
||||
|
||||
```
|
||||
...
|
||||
environment:
|
||||
SERVICE_BINDING_ROOT: /bindings
|
||||
...
|
||||
volumes:
|
||||
- "/path/to/bindings/ca-certificates/:/bindings/ca-certificates/:ro"
|
||||
...
|
||||
```
|
||||
|
||||
## Deployment
|
||||
*Ausführen als Docker Conatiner:*
|
||||
*Ausführen als Docker Container:*
|
||||
|
||||
```bash
|
||||
cd ./deploy
|
||||
@ -154,6 +267,50 @@ Wenn gewünscht, Änderungen in der `.env` vornehmen.
|
||||
docker compose up -d
|
||||
```
|
||||
|
||||
### Einfaches Beispiel für ein eigenes Docker-Compose-File
|
||||
|
||||
Die Datei [`docs/docker-compose.yml`](docs/docker-compose.yml) zeigt eine einfache Konfiguration für REST-Requests basierend
|
||||
auf Docker-Compose mit der gestartet werden kann.
|
||||
|
||||
### Betrieb hinter einem Reverse-Proxy
|
||||
|
||||
Die Anwendung verarbeitet `X-Forwarded`-HTTP-Header und kann daher auch hinter einem Reverse-Proxy betrieben werden.
|
||||
|
||||
Dabei werden, je nachdem welche Header durch den Reverse-Proxy gesendet werden auch Protokoll, Host oder auch Path-Prefix
|
||||
automatisch erkannt und verwendet werden. Dadurch ist z.B. eine abweichende Angabe des Pfads problemlos möglich.
|
||||
|
||||
#### Beispiel *Traefik* (mit Docker-Labels):
|
||||
|
||||
Das folgende Beispiel zeigt die Konfiguration in einer Docker-Compose-Datei mit Service-Labels.
|
||||
|
||||
```
|
||||
...
|
||||
deploy:
|
||||
labels:
|
||||
- "traefik.http.routers.etl.rule=PathPrefix(`/etl-processor`)"
|
||||
- "traefik.http.routers.etl.middlewares=etl-path-strip"
|
||||
- "traefik.http.middlewares.etl-path-strip.stripprefix.prefixes=/etl-processor"
|
||||
...
|
||||
```
|
||||
|
||||
#### Beispiel *nginx*
|
||||
|
||||
Das folgende Beispiel zeigt die Konfiguration einer _location_ in einer nginx-Konfigurationsdatei.
|
||||
|
||||
```
|
||||
...
|
||||
location /etl-processor {
|
||||
set $upstream http://<beispiel:8080>/;
|
||||
proxy_set_header Host $host;
|
||||
proxy_set_header X-Forwarded-Scheme $scheme;
|
||||
proxy_set_header X-Forwarded-Proto $scheme;
|
||||
proxy_set_header X-Forwarded-For $remote_addr;
|
||||
proxy_set_header X-Real-IP $remote_addr;
|
||||
proxy_pass $upstream;
|
||||
}
|
||||
...
|
||||
```
|
||||
|
||||
## Entwicklungssetup
|
||||
|
||||
Zum Starten einer lokalen Entwicklungs- und Testumgebung kann die beiliegende Datei `dev-compose.yml` verwendet werden.
|
||||
|
5
bindings/README.md
Normal file
5
bindings/README.md
Normal file
@ -0,0 +1,5 @@
|
||||
# Hinweis für Root CA Zertifikate
|
||||
|
||||
PEM-Datei(en) in das Verzeichnis `ca-certificates` ablegen.
|
||||
|
||||
Die Datei `type` gibt dabei an, dass hier CA Zertifikate zu finden sind.
|
1
bindings/ca-certificates/type
Normal file
1
bindings/ca-certificates/type
Normal file
@ -0,0 +1 @@
|
||||
ca-certificates
|
@ -4,20 +4,23 @@ import org.springframework.boot.gradle.tasks.bundling.BootBuildImage
|
||||
|
||||
plugins {
|
||||
war
|
||||
id("org.springframework.boot") version "3.2.1"
|
||||
id("org.springframework.boot") version "3.2.2"
|
||||
id("io.spring.dependency-management") version "1.1.4"
|
||||
kotlin("jvm") version "1.9.22"
|
||||
kotlin("plugin.spring") version "1.9.22"
|
||||
}
|
||||
|
||||
group = "de.ukw.ccc"
|
||||
version = "0.4.0"
|
||||
version = "0.7.0"
|
||||
|
||||
var versions = mapOf(
|
||||
"bwhc-dto-java" to "0.2.0",
|
||||
"hapi-fhir" to "6.10.2",
|
||||
"httpclient5" to "5.2.1",
|
||||
"mockito-kotlin" to "5.2.1"
|
||||
"mockito-kotlin" to "5.2.1",
|
||||
// Webjars
|
||||
"echarts" to "5.4.3",
|
||||
"htmx.org" to "1.9.10"
|
||||
)
|
||||
|
||||
java {
|
||||
@ -54,6 +57,9 @@ dependencies {
|
||||
implementation("org.springframework.boot:spring-boot-starter-thymeleaf")
|
||||
implementation("org.springframework.boot:spring-boot-starter-web")
|
||||
implementation("org.springframework.boot:spring-boot-starter-data-jdbc")
|
||||
implementation("org.springframework.boot:spring-boot-starter-security")
|
||||
implementation("org.springframework.boot:spring-boot-starter-oauth2-client")
|
||||
implementation("org.thymeleaf.extras:thymeleaf-extras-springsecurity6")
|
||||
implementation("com.fasterxml.jackson.module:jackson-module-kotlin")
|
||||
implementation("org.springframework.kafka:spring-kafka")
|
||||
implementation("org.flywaydb:flyway-mysql")
|
||||
@ -64,6 +70,9 @@ dependencies {
|
||||
implementation("ca.uhn.hapi.fhir:hapi-fhir-structures-r4:${versions["hapi-fhir"]}")
|
||||
implementation("org.apache.httpcomponents.client5:httpclient5:${versions["httpclient5"]}")
|
||||
implementation("com.jayway.jsonpath:json-path")
|
||||
implementation("org.webjars:webjars-locator:0.50")
|
||||
implementation("org.webjars.npm:echarts:${versions["echarts"]}")
|
||||
implementation("org.webjars.npm:htmx.org:${versions["htmx.org"]}")
|
||||
runtimeOnly("org.mariadb.jdbc:mariadb-java-client")
|
||||
runtimeOnly("org.postgresql:postgresql")
|
||||
developmentOnly("org.springframework.boot:spring-boot-devtools")
|
||||
@ -71,6 +80,7 @@ dependencies {
|
||||
annotationProcessor("org.springframework.boot:spring-boot-configuration-processor")
|
||||
providedRuntime("org.springframework.boot:spring-boot-starter-tomcat")
|
||||
testImplementation("org.springframework.boot:spring-boot-starter-test")
|
||||
testImplementation("org.springframework.security:spring-security-test")
|
||||
testImplementation("io.projectreactor:reactor-test")
|
||||
testImplementation("org.mockito.kotlin:mockito-kotlin:${versions["mockito-kotlin"]}")
|
||||
integrationTestImplementation("org.testcontainers:junit-jupiter")
|
||||
@ -103,7 +113,14 @@ task<Test>("integrationTest") {
|
||||
tasks.named<BootBuildImage>("bootBuildImage") {
|
||||
imageName.set("ghcr.io/ccc-mf/etl-processor")
|
||||
|
||||
// Binding for CA Certs
|
||||
bindings.set(listOf(
|
||||
"$rootDir/bindings/ca-certificates/:/platform/bindings/ca-certificates"
|
||||
))
|
||||
|
||||
environment.set(environment.get() + mapOf(
|
||||
// Enable this line to embed CA Certs into image on build time
|
||||
//"BP_EMBED_CERTS" to "true",
|
||||
"BP_OCI_SOURCE" to "https://github.com/CCC-MF/etl-processor",
|
||||
"BP_OCI_LICENSES" to "AGPLv3",
|
||||
"BP_OCI_DESCRIPTION" to "ETL Processor for bwHC MTB files"
|
||||
|
@ -18,6 +18,8 @@ services:
|
||||
APP_KAFKA_GROUP_ID: ${DNPM_KAFKA_GROUP_ID}
|
||||
APP_KAFKA_RESPONSE_TOPIC: ${DNPM_KAFKA_RESPONSE_TOPIC}
|
||||
APP_REST_URI: ${DNPM_BWHC_REST_URI}
|
||||
APP_SECURITY_ADMIN_USER: ${DNPM_ADMIN_USER}
|
||||
APP_SECURITY_ADMIN_PASSWORD: ${DNPM_ADMIN_PASSWORD}
|
||||
SPRING_DATASOURCE_URL: ${DNPM_DATASOURCE_URL}
|
||||
SPRING_DATASOURCE_PASSWORD: ${DNPM_MARIADB_USER_PW}
|
||||
SPRING_DATASOURCE_USERNAME: ${DNPM_MARIADB_DB}
|
||||
|
@ -2,6 +2,10 @@
|
||||
DNPM_MONITORING_HTTP_PORT=8088
|
||||
DNPM_LOG_LEVEL=INFO
|
||||
|
||||
# ADMIN USER CREDENTIALS
|
||||
DNPM_ADMIN_USER=admin
|
||||
DNPM_ADMIN_PASSWORD=
|
||||
|
||||
# GPAS or BUILDIN
|
||||
DNPM_PSEUDONYMIZE_GENERATOR=BUILDIN
|
||||
DNPM_APP_PSEUDONYMIZE_PREFIX=ANONYM
|
||||
|
26
docs/docker-compose.yml
Normal file
26
docs/docker-compose.yml
Normal file
@ -0,0 +1,26 @@
|
||||
### Example for docker-compose
|
||||
version: '3.7'
|
||||
|
||||
volumes:
|
||||
data:
|
||||
|
||||
services:
|
||||
|
||||
### ETL-Processor
|
||||
etl-processor:
|
||||
image: ghcr.io/ccc-mf/etl-processor:latest
|
||||
environment:
|
||||
APP_REST_URI: http://bwhc-backend/bwhc/etl/api
|
||||
SPRING_DATASOURCE_URL: jdbc:postgresql://postgres/etl
|
||||
SPRING_DATASOURCE_USERNAME: etl
|
||||
SPRING_DATASOURCE_PASSWORD: etl-password
|
||||
|
||||
### Database
|
||||
postgres:
|
||||
image: postgres:alpine
|
||||
environment:
|
||||
POSTGRES_DB: etl
|
||||
POSTGRES_USER: etl
|
||||
POSTGRES_PASSWORD: etl-password
|
||||
volumes:
|
||||
- data:/var/lib/postgresql/data
|
BIN
docs/login.png
Normal file
BIN
docs/login.png
Normal file
Binary file not shown.
After Width: | Height: | Size: 9.2 KiB |
BIN
docs/tokens.png
Normal file
BIN
docs/tokens.png
Normal file
Binary file not shown.
After Width: | Height: | Size: 30 KiB |
@ -48,7 +48,8 @@ import org.testcontainers.junit.jupiter.Testcontainers
|
||||
@MockBean(MtbFileSender::class)
|
||||
@TestPropertySource(
|
||||
properties = [
|
||||
"app.rest.uri=http://example.com"
|
||||
"app.rest.uri=http://example.com",
|
||||
"app.pseudonymize.generator=buildin"
|
||||
]
|
||||
)
|
||||
class EtlProcessorApplicationTests : AbstractTestcontainerTest() {
|
||||
@ -64,6 +65,7 @@ class EtlProcessorApplicationTests : AbstractTestcontainerTest() {
|
||||
@AutoConfigureMockMvc
|
||||
@TestPropertySource(
|
||||
properties = [
|
||||
"app.pseudonymize.generator=buildin",
|
||||
"app.transformations[0].path=diagnoses[*].icd10.version",
|
||||
"app.transformations[0].from=2013",
|
||||
"app.transformations[0].to=2014",
|
||||
|
@ -1,7 +1,7 @@
|
||||
/*
|
||||
* This file is part of ETL-Processor
|
||||
*
|
||||
* Copyright (c) 2023 Comprehensive Cancer Center Mainfranken, Datenintegrationszentrum Philipps-Universität Marburg and Contributors
|
||||
* Copyright (c) 2024 Comprehensive Cancer Center Mainfranken, Datenintegrationszentrum Philipps-Universität Marburg and Contributors
|
||||
*
|
||||
* This program is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
@ -23,6 +23,10 @@ import com.fasterxml.jackson.databind.ObjectMapper
|
||||
import dev.dnpm.etl.processor.monitoring.RequestRepository
|
||||
import dev.dnpm.etl.processor.output.KafkaMtbFileSender
|
||||
import dev.dnpm.etl.processor.output.RestMtbFileSender
|
||||
import dev.dnpm.etl.processor.pseudonym.AnonymizingGenerator
|
||||
import dev.dnpm.etl.processor.pseudonym.GpasPseudonymGenerator
|
||||
import dev.dnpm.etl.processor.services.TokenRepository
|
||||
import dev.dnpm.etl.processor.services.TokenService
|
||||
import org.assertj.core.api.Assertions.assertThat
|
||||
import org.junit.jupiter.api.Nested
|
||||
import org.junit.jupiter.api.Test
|
||||
@ -31,13 +35,27 @@ import org.springframework.beans.factory.NoSuchBeanDefinitionException
|
||||
import org.springframework.boot.autoconfigure.kafka.KafkaAutoConfiguration
|
||||
import org.springframework.boot.test.context.SpringBootTest
|
||||
import org.springframework.boot.test.mock.mockito.MockBean
|
||||
import org.springframework.boot.test.mock.mockito.MockBeans
|
||||
import org.springframework.context.ApplicationContext
|
||||
import org.springframework.security.crypto.password.PasswordEncoder
|
||||
import org.springframework.security.provisioning.InMemoryUserDetailsManager
|
||||
import org.springframework.test.context.ContextConfiguration
|
||||
import org.springframework.test.context.TestPropertySource
|
||||
|
||||
@SpringBootTest
|
||||
@ContextConfiguration(classes = [AppConfiguration::class, KafkaAutoConfiguration::class, AppKafkaConfiguration::class, AppRestConfiguration::class])
|
||||
@ContextConfiguration(classes = [
|
||||
AppConfiguration::class,
|
||||
AppSecurityConfiguration::class,
|
||||
KafkaAutoConfiguration::class,
|
||||
AppKafkaConfiguration::class,
|
||||
AppRestConfiguration::class
|
||||
])
|
||||
@MockBean(ObjectMapper::class)
|
||||
@TestPropertySource(
|
||||
properties = [
|
||||
"app.pseudonymize.generator=BUILDIN",
|
||||
]
|
||||
)
|
||||
class AppConfigurationTest {
|
||||
|
||||
@Nested
|
||||
@ -116,4 +134,108 @@ class AppConfigurationTest {
|
||||
|
||||
}
|
||||
|
||||
@Nested
|
||||
inner class AppConfigurationPseudonymizeTest {
|
||||
|
||||
@Nested
|
||||
@TestPropertySource(
|
||||
properties = [
|
||||
"app.pseudonymize.generator=",
|
||||
"app.pseudonymizer=buildin",
|
||||
]
|
||||
)
|
||||
inner class AppConfigurationPseudonymizerBuildinTest(private val context: ApplicationContext) {
|
||||
|
||||
@Test
|
||||
fun shouldUseConfiguredGenerator() {
|
||||
assertThat(context.getBean(AnonymizingGenerator::class.java)).isNotNull
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
@Nested
|
||||
@TestPropertySource(
|
||||
properties = [
|
||||
"app.pseudonymize.generator=",
|
||||
"app.pseudonymizer=gpas",
|
||||
]
|
||||
)
|
||||
inner class AppConfigurationPseudonymizerGpasTest(private val context: ApplicationContext) {
|
||||
|
||||
@Test
|
||||
fun shouldUseConfiguredGenerator() {
|
||||
assertThat(context.getBean(GpasPseudonymGenerator::class.java)).isNotNull
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
@Nested
|
||||
@TestPropertySource(
|
||||
properties = [
|
||||
"app.pseudonymize.generator=buildin",
|
||||
"app.pseudonymizer=",
|
||||
]
|
||||
)
|
||||
inner class AppConfigurationPseudonymizeGeneratorBuildinTest(private val context: ApplicationContext) {
|
||||
|
||||
@Test
|
||||
fun shouldUseConfiguredGenerator() {
|
||||
assertThat(context.getBean(AnonymizingGenerator::class.java)).isNotNull
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
@Nested
|
||||
@TestPropertySource(
|
||||
properties = [
|
||||
"app.pseudonymize.generator=gpas",
|
||||
"app.pseudonymizer=",
|
||||
]
|
||||
)
|
||||
inner class AppConfigurationPseudonymizeGeneratorGpasTest(private val context: ApplicationContext) {
|
||||
|
||||
@Test
|
||||
fun shouldUseConfiguredGenerator() {
|
||||
assertThat(context.getBean(GpasPseudonymGenerator::class.java)).isNotNull
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
@Nested
|
||||
@TestPropertySource(
|
||||
properties = [
|
||||
"app.security.enable-tokens=true"
|
||||
]
|
||||
)
|
||||
@MockBeans(value = [
|
||||
MockBean(InMemoryUserDetailsManager::class),
|
||||
MockBean(PasswordEncoder::class),
|
||||
MockBean(TokenRepository::class)
|
||||
])
|
||||
inner class AppConfigurationTokenEnabledTest(private val context: ApplicationContext) {
|
||||
|
||||
@Test
|
||||
fun checkTokenService() {
|
||||
assertThat(context.getBean(TokenService::class.java)).isNotNull
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
@Nested
|
||||
@MockBeans(value = [
|
||||
MockBean(InMemoryUserDetailsManager::class),
|
||||
MockBean(PasswordEncoder::class),
|
||||
MockBean(TokenRepository::class)
|
||||
])
|
||||
inner class AppConfigurationTokenDisabledTest(private val context: ApplicationContext) {
|
||||
|
||||
@Test
|
||||
fun checkTokenService() {
|
||||
assertThrows<NoSuchBeanDefinitionException> { context.getBean(TokenService::class.java) }
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
}
|
@ -46,6 +46,7 @@ import java.util.*
|
||||
@MockBean(MtbFileSender::class)
|
||||
@TestPropertySource(
|
||||
properties = [
|
||||
"app.pseudonymize.generator=buildin",
|
||||
"app.rest.uri=http://example.com"
|
||||
]
|
||||
)
|
||||
|
@ -0,0 +1,110 @@
|
||||
/*
|
||||
* This file is part of ETL-Processor
|
||||
*
|
||||
* Copyright (c) 2024 Comprehensive Cancer Center Mainfranken, Datenintegrationszentrum Philipps-Universität Marburg and Contributors
|
||||
*
|
||||
* This program is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
* by the Free Software Foundation, either version 3 of the License, or
|
||||
* (at your option) any later version.
|
||||
*
|
||||
* This program is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
* GNU Affero General Public License for more details.
|
||||
*
|
||||
* You should have received a copy of the GNU Affero General Public License
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
*/
|
||||
|
||||
package dev.dnpm.etl.processor.web
|
||||
|
||||
import dev.dnpm.etl.processor.config.AppSecurityConfiguration
|
||||
import dev.dnpm.etl.processor.monitoring.ConnectionCheckService
|
||||
import dev.dnpm.etl.processor.output.MtbFileSender
|
||||
import dev.dnpm.etl.processor.pseudonym.Generator
|
||||
import dev.dnpm.etl.processor.services.RequestProcessor
|
||||
import dev.dnpm.etl.processor.services.TransformationService
|
||||
import org.junit.jupiter.api.BeforeEach
|
||||
import org.junit.jupiter.api.Test
|
||||
import org.junit.jupiter.api.extension.ExtendWith
|
||||
import org.mockito.junit.jupiter.MockitoExtension
|
||||
import org.springframework.beans.factory.annotation.Autowired
|
||||
import org.springframework.boot.test.autoconfigure.web.servlet.WebMvcTest
|
||||
import org.springframework.boot.test.mock.mockito.MockBean
|
||||
import org.springframework.http.HttpHeaders
|
||||
import org.springframework.http.MediaType
|
||||
import org.springframework.security.test.web.servlet.request.SecurityMockMvcRequestPostProcessors.anonymous
|
||||
import org.springframework.security.test.web.servlet.request.SecurityMockMvcRequestPostProcessors.user
|
||||
import org.springframework.test.context.ContextConfiguration
|
||||
import org.springframework.test.context.TestPropertySource
|
||||
import org.springframework.test.context.junit.jupiter.SpringExtension
|
||||
import org.springframework.test.web.servlet.MockMvc
|
||||
import org.springframework.test.web.servlet.get
|
||||
import reactor.core.publisher.Sinks
|
||||
|
||||
abstract class MockSink : Sinks.Many<Boolean>
|
||||
|
||||
@WebMvcTest(controllers = [ConfigController::class])
|
||||
@ExtendWith(value = [MockitoExtension::class, SpringExtension::class])
|
||||
@ContextConfiguration(
|
||||
classes = [
|
||||
ConfigController::class,
|
||||
AppSecurityConfiguration::class
|
||||
]
|
||||
)
|
||||
@TestPropertySource(
|
||||
properties = [
|
||||
"app.pseudonymize.generator=BUILDIN",
|
||||
"app.security.admin-user=admin",
|
||||
"app.security.admin-password={noop}very-secret",
|
||||
"app.security.enable-tokens=true"
|
||||
]
|
||||
)
|
||||
@MockBean(name = "configsUpdateProducer", classes = [MockSink::class])
|
||||
@MockBean(
|
||||
Generator::class,
|
||||
MtbFileSender::class,
|
||||
ConnectionCheckService::class,
|
||||
RequestProcessor::class,
|
||||
TransformationService::class
|
||||
)
|
||||
class ConfigControllerTest {
|
||||
|
||||
private lateinit var mockMvc: MockMvc
|
||||
|
||||
private lateinit var requestProcessor: RequestProcessor
|
||||
|
||||
@BeforeEach
|
||||
fun setup(
|
||||
@Autowired mockMvc: MockMvc,
|
||||
@Autowired requestProcessor: RequestProcessor
|
||||
) {
|
||||
this.mockMvc = mockMvc
|
||||
this.requestProcessor = requestProcessor
|
||||
}
|
||||
|
||||
@Test
|
||||
fun testShouldShowConfigPageIfLoggedIn() {
|
||||
mockMvc.get("/configs") {
|
||||
with(user("admin").roles("ADMIN"))
|
||||
accept(MediaType.TEXT_HTML)
|
||||
}.andExpect {
|
||||
status { isOk() }
|
||||
}
|
||||
}
|
||||
|
||||
@Test
|
||||
fun testShouldRedirectToLoginPageIfNotLoggedIn() {
|
||||
mockMvc.get("/configs") {
|
||||
with(anonymous())
|
||||
accept(MediaType.TEXT_HTML)
|
||||
}.andExpect {
|
||||
status { isFound() }
|
||||
header {
|
||||
stringValues(HttpHeaders.LOCATION, "http://localhost/login")
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
}
|
@ -0,0 +1,157 @@
|
||||
/*
|
||||
* This file is part of ETL-Processor
|
||||
*
|
||||
* Copyright (c) 2024 Comprehensive Cancer Center Mainfranken, Datenintegrationszentrum Philipps-Universität Marburg and Contributors
|
||||
*
|
||||
* This program is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
* by the Free Software Foundation, either version 3 of the License, or
|
||||
* (at your option) any later version.
|
||||
*
|
||||
* This program is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
* GNU Affero General Public License for more details.
|
||||
*
|
||||
* You should have received a copy of the GNU Affero General Public License
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
*/
|
||||
|
||||
package dev.dnpm.etl.processor.web
|
||||
|
||||
import com.fasterxml.jackson.databind.ObjectMapper
|
||||
import de.ukw.ccc.bwhc.dto.*
|
||||
import dev.dnpm.etl.processor.config.AppSecurityConfiguration
|
||||
import dev.dnpm.etl.processor.services.RequestProcessor
|
||||
import dev.dnpm.etl.processor.services.TokenRepository
|
||||
import org.junit.jupiter.api.BeforeEach
|
||||
import org.junit.jupiter.api.Test
|
||||
import org.junit.jupiter.api.extension.ExtendWith
|
||||
import org.mockito.ArgumentMatchers.anyString
|
||||
import org.mockito.junit.jupiter.MockitoExtension
|
||||
import org.mockito.kotlin.any
|
||||
import org.mockito.kotlin.never
|
||||
import org.mockito.kotlin.times
|
||||
import org.mockito.kotlin.verify
|
||||
import org.springframework.beans.factory.annotation.Autowired
|
||||
import org.springframework.boot.test.autoconfigure.web.servlet.WebMvcTest
|
||||
import org.springframework.boot.test.mock.mockito.MockBean
|
||||
import org.springframework.http.MediaType
|
||||
import org.springframework.security.test.web.servlet.request.SecurityMockMvcRequestPostProcessors.anonymous
|
||||
import org.springframework.security.test.web.servlet.request.SecurityMockMvcRequestPostProcessors.user
|
||||
import org.springframework.test.context.ContextConfiguration
|
||||
import org.springframework.test.context.TestPropertySource
|
||||
import org.springframework.test.context.junit.jupiter.SpringExtension
|
||||
import org.springframework.test.web.servlet.MockMvc
|
||||
import org.springframework.test.web.servlet.delete
|
||||
import org.springframework.test.web.servlet.post
|
||||
|
||||
@WebMvcTest(controllers = [MtbFileRestController::class])
|
||||
@ExtendWith(value = [MockitoExtension::class, SpringExtension::class])
|
||||
@ContextConfiguration(
|
||||
classes = [
|
||||
MtbFileRestController::class,
|
||||
AppSecurityConfiguration::class
|
||||
]
|
||||
)
|
||||
@MockBean(TokenRepository::class, RequestProcessor::class)
|
||||
@TestPropertySource(
|
||||
properties = [
|
||||
"app.pseudonymize.generator=BUILDIN",
|
||||
"app.security.admin-user=admin",
|
||||
"app.security.admin-password={noop}very-secret",
|
||||
"app.security.enable-tokens=true"
|
||||
]
|
||||
)
|
||||
class MtbFileRestControllerTest {
|
||||
|
||||
private lateinit var mockMvc: MockMvc
|
||||
|
||||
private lateinit var requestProcessor: RequestProcessor
|
||||
|
||||
@BeforeEach
|
||||
fun setup(
|
||||
@Autowired mockMvc: MockMvc,
|
||||
@Autowired requestProcessor: RequestProcessor
|
||||
) {
|
||||
this.mockMvc = mockMvc
|
||||
this.requestProcessor = requestProcessor
|
||||
}
|
||||
|
||||
@Test
|
||||
fun testShouldGrantPermissionToSendMtbFile() {
|
||||
mockMvc.post("/mtbfile") {
|
||||
with(user("onkostarserver").roles("MTBFILE"))
|
||||
contentType = MediaType.APPLICATION_JSON
|
||||
content = ObjectMapper().writeValueAsString(mtbFile)
|
||||
}.andExpect {
|
||||
status { isAccepted() }
|
||||
}
|
||||
|
||||
verify(requestProcessor, times(1)).processMtbFile(any())
|
||||
}
|
||||
|
||||
@Test
|
||||
fun testShouldDenyPermissionToSendMtbFile() {
|
||||
mockMvc.post("/mtbfile") {
|
||||
with(anonymous())
|
||||
contentType = MediaType.APPLICATION_JSON
|
||||
content = ObjectMapper().writeValueAsString(mtbFile)
|
||||
}.andExpect {
|
||||
status { isUnauthorized() }
|
||||
}
|
||||
|
||||
verify(requestProcessor, never()).processMtbFile(any())
|
||||
}
|
||||
|
||||
@Test
|
||||
fun testShouldGrantPermissionToDeletePatientData() {
|
||||
mockMvc.delete("/mtbfile/12345678") {
|
||||
with(user("onkostarserver").roles("MTBFILE"))
|
||||
}.andExpect {
|
||||
status { isAccepted() }
|
||||
}
|
||||
|
||||
verify(requestProcessor, times(1)).processDeletion(anyString())
|
||||
}
|
||||
|
||||
@Test
|
||||
fun testShouldDenyPermissionToDeletePatientData() {
|
||||
mockMvc.delete("/mtbfile/12345678") {
|
||||
with(anonymous())
|
||||
}.andExpect {
|
||||
status { isUnauthorized() }
|
||||
}
|
||||
|
||||
verify(requestProcessor, never()).processDeletion(anyString())
|
||||
}
|
||||
|
||||
companion object {
|
||||
|
||||
val mtbFile: MtbFile = MtbFile.builder()
|
||||
.withPatient(
|
||||
Patient.builder()
|
||||
.withId("PID")
|
||||
.withBirthDate("2000-08-08")
|
||||
.withGender(Patient.Gender.MALE)
|
||||
.build()
|
||||
)
|
||||
.withConsent(
|
||||
Consent.builder()
|
||||
.withId("1")
|
||||
.withStatus(Consent.Status.ACTIVE)
|
||||
.withPatient("PID")
|
||||
.build()
|
||||
)
|
||||
.withEpisode(
|
||||
Episode.builder()
|
||||
.withId("1")
|
||||
.withPatient("PID")
|
||||
.withPeriod(PeriodStart("2023-08-08"))
|
||||
.build()
|
||||
)
|
||||
.build()
|
||||
|
||||
}
|
||||
|
||||
}
|
@ -20,9 +20,10 @@
|
||||
package dev.dnpm.etl.processor
|
||||
|
||||
import org.springframework.boot.autoconfigure.SpringBootApplication
|
||||
import org.springframework.boot.autoconfigure.security.servlet.SecurityAutoConfiguration
|
||||
import org.springframework.boot.runApplication
|
||||
|
||||
@SpringBootApplication
|
||||
@SpringBootApplication(exclude = [SecurityAutoConfiguration::class])
|
||||
class EtlProcessorApplication
|
||||
|
||||
fun main(args: Array<String>) {
|
||||
|
@ -1,7 +1,7 @@
|
||||
/*
|
||||
* This file is part of ETL-Processor
|
||||
*
|
||||
* Copyright (c) 2023 Comprehensive Cancer Center Mainfranken, Datenintegrationszentrum Philipps-Universität Marburg and Contributors
|
||||
* Copyright (c) 2024 Comprehensive Cancer Center Mainfranken, Datenintegrationszentrum Philipps-Universität Marburg and Contributors
|
||||
*
|
||||
* This program is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
@ -20,11 +20,16 @@
|
||||
package dev.dnpm.etl.processor.config
|
||||
|
||||
import org.springframework.boot.context.properties.ConfigurationProperties
|
||||
import org.springframework.boot.context.properties.DeprecatedConfigurationProperty
|
||||
|
||||
@ConfigurationProperties(AppConfigProperties.NAME)
|
||||
data class AppConfigProperties(
|
||||
var bwhcUri: String?,
|
||||
var generator: PseudonymGenerator = PseudonymGenerator.BUILDIN,
|
||||
@get:DeprecatedConfigurationProperty(
|
||||
reason = "Deprecated in favor of 'app.pseudonymize.generator'",
|
||||
replacement = "app.pseudonymize.generator"
|
||||
)
|
||||
var pseudonymizer: PseudonymGenerator = PseudonymGenerator.BUILDIN,
|
||||
var transformations: List<TransformationProperties> = listOf()
|
||||
) {
|
||||
companion object {
|
||||
@ -34,6 +39,7 @@ data class AppConfigProperties(
|
||||
|
||||
@ConfigurationProperties(PseudonymizeConfigProperties.NAME)
|
||||
data class PseudonymizeConfigProperties(
|
||||
var generator: PseudonymGenerator = PseudonymGenerator.BUILDIN,
|
||||
val prefix: String = "UNKNOWN",
|
||||
) {
|
||||
companion object {
|
||||
@ -76,6 +82,18 @@ data class KafkaTargetProperties(
|
||||
}
|
||||
}
|
||||
|
||||
@ConfigurationProperties(SecurityConfigProperties.NAME)
|
||||
data class SecurityConfigProperties(
|
||||
val adminUser: String?,
|
||||
val adminPassword: String?,
|
||||
val enableTokens: Boolean = false,
|
||||
val enableOidc: Boolean = false
|
||||
) {
|
||||
companion object {
|
||||
const val NAME = "app.security"
|
||||
}
|
||||
}
|
||||
|
||||
enum class PseudonymGenerator {
|
||||
BUILDIN,
|
||||
GPAS
|
||||
|
@ -25,15 +25,27 @@ import dev.dnpm.etl.processor.pseudonym.AnonymizingGenerator
|
||||
import dev.dnpm.etl.processor.pseudonym.Generator
|
||||
import dev.dnpm.etl.processor.pseudonym.GpasPseudonymGenerator
|
||||
import dev.dnpm.etl.processor.pseudonym.PseudonymizeService
|
||||
import dev.dnpm.etl.processor.services.TokenRepository
|
||||
import dev.dnpm.etl.processor.services.TokenService
|
||||
import dev.dnpm.etl.processor.services.Transformation
|
||||
import dev.dnpm.etl.processor.services.TransformationService
|
||||
import org.slf4j.LoggerFactory
|
||||
import org.springframework.boot.autoconfigure.condition.ConditionalOnMissingBean
|
||||
import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty
|
||||
import org.springframework.boot.context.properties.EnableConfigurationProperties
|
||||
import org.springframework.context.annotation.Bean
|
||||
import org.springframework.context.annotation.Configuration
|
||||
import org.springframework.retry.policy.SimpleRetryPolicy
|
||||
import org.springframework.retry.support.RetryTemplate
|
||||
import org.springframework.retry.support.RetryTemplateBuilder
|
||||
import org.springframework.scheduling.annotation.EnableScheduling
|
||||
import org.springframework.security.crypto.password.PasswordEncoder
|
||||
import org.springframework.security.provisioning.InMemoryUserDetailsManager
|
||||
import org.springframework.security.provisioning.UserDetailsManager
|
||||
import reactor.core.publisher.Sinks
|
||||
import kotlin.time.Duration.Companion.seconds
|
||||
import kotlin.time.toJavaDuration
|
||||
|
||||
|
||||
@Configuration
|
||||
@EnableConfigurationProperties(
|
||||
@ -48,18 +60,32 @@ class AppConfiguration {
|
||||
|
||||
private val logger = LoggerFactory.getLogger(AppConfiguration::class.java)
|
||||
|
||||
@ConditionalOnProperty(value = ["app.pseudonymizer"], havingValue = "GPAS")
|
||||
@ConditionalOnProperty(value = ["app.pseudonymize.generator"], havingValue = "GPAS")
|
||||
@Bean
|
||||
fun gpasPseudonymGenerator(configProperties: GPasConfigProperties): Generator {
|
||||
return GpasPseudonymGenerator(configProperties)
|
||||
}
|
||||
|
||||
@ConditionalOnProperty(value = ["app.pseudonymizer"], havingValue = "BUILDIN", matchIfMissing = true)
|
||||
@ConditionalOnProperty(value = ["app.pseudonymize.generator"], havingValue = "BUILDIN", matchIfMissing = true)
|
||||
@Bean
|
||||
fun buildinPseudonymGenerator(): Generator {
|
||||
return AnonymizingGenerator()
|
||||
}
|
||||
|
||||
@ConditionalOnProperty(value = ["app.pseudonymizer"], havingValue = "GPAS")
|
||||
@ConditionalOnMissingBean
|
||||
@Bean
|
||||
fun gpasPseudonymGeneratorOnDeprecatedProperty(configProperties: GPasConfigProperties): Generator {
|
||||
return GpasPseudonymGenerator(configProperties)
|
||||
}
|
||||
|
||||
@ConditionalOnProperty(value = ["app.pseudonymizer"], havingValue = "BUILDIN")
|
||||
@ConditionalOnMissingBean
|
||||
@Bean
|
||||
fun buildinPseudonymGeneratorOnDeprecatedProperty(): Generator {
|
||||
return AnonymizingGenerator()
|
||||
}
|
||||
|
||||
@Bean
|
||||
fun pseudonymizeService(
|
||||
generator: Generator,
|
||||
@ -73,11 +99,6 @@ class AppConfiguration {
|
||||
return ReportService(objectMapper)
|
||||
}
|
||||
|
||||
@Bean
|
||||
fun statisticsUpdateProducer(): Sinks.Many<Any> {
|
||||
return Sinks.many().multicast().directBestEffort()
|
||||
}
|
||||
|
||||
@Bean
|
||||
fun transformationService(
|
||||
objectMapper: ObjectMapper,
|
||||
@ -89,5 +110,30 @@ class AppConfiguration {
|
||||
})
|
||||
}
|
||||
|
||||
@Bean
|
||||
fun retryTemplate(): RetryTemplate {
|
||||
return RetryTemplateBuilder()
|
||||
.notRetryOn(IllegalArgumentException::class.java)
|
||||
.fixedBackoff(5.seconds.toJavaDuration())
|
||||
.customPolicy(SimpleRetryPolicy(3))
|
||||
.build()
|
||||
}
|
||||
|
||||
@ConditionalOnProperty(value = ["app.security.enable-tokens"], havingValue = "true")
|
||||
@Bean
|
||||
fun tokenService(userDetailsManager: InMemoryUserDetailsManager, passwordEncoder: PasswordEncoder, tokenRepository: TokenRepository): TokenService {
|
||||
return TokenService(userDetailsManager, passwordEncoder, tokenRepository)
|
||||
}
|
||||
|
||||
@Bean
|
||||
fun statisticsUpdateProducer(): Sinks.Many<Any> {
|
||||
return Sinks.many().multicast().directBestEffort()
|
||||
}
|
||||
|
||||
@Bean
|
||||
fun configsUpdateProducer(): Sinks.Many<Boolean> {
|
||||
return Sinks.many().multicast().directBestEffort()
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
|
@ -37,6 +37,8 @@ import org.springframework.kafka.core.ConsumerFactory
|
||||
import org.springframework.kafka.core.KafkaTemplate
|
||||
import org.springframework.kafka.listener.ContainerProperties
|
||||
import org.springframework.kafka.listener.KafkaMessageListenerContainer
|
||||
import org.springframework.retry.support.RetryTemplate
|
||||
import reactor.core.publisher.Sinks
|
||||
|
||||
@Configuration
|
||||
@EnableConfigurationProperties(
|
||||
@ -53,10 +55,11 @@ class AppKafkaConfiguration {
|
||||
fun kafkaMtbFileSender(
|
||||
kafkaTemplate: KafkaTemplate<String, String>,
|
||||
kafkaTargetProperties: KafkaTargetProperties,
|
||||
retryTemplate: RetryTemplate,
|
||||
objectMapper: ObjectMapper
|
||||
): MtbFileSender {
|
||||
logger.info("Selected 'KafkaMtbFileSender'")
|
||||
return KafkaMtbFileSender(kafkaTemplate, kafkaTargetProperties, objectMapper)
|
||||
return KafkaMtbFileSender(kafkaTemplate, kafkaTargetProperties, retryTemplate, objectMapper)
|
||||
}
|
||||
|
||||
@Bean
|
||||
@ -79,8 +82,8 @@ class AppKafkaConfiguration {
|
||||
}
|
||||
|
||||
@Bean
|
||||
fun connectionCheckService(consumerFactory: ConsumerFactory<String, String>): ConnectionCheckService {
|
||||
return KafkaConnectionCheckService(consumerFactory.createConsumer())
|
||||
fun connectionCheckService(consumerFactory: ConsumerFactory<String, String>, configsUpdateProducer: Sinks.Many<Boolean>): ConnectionCheckService {
|
||||
return KafkaConnectionCheckService(consumerFactory.createConsumer(), configsUpdateProducer)
|
||||
}
|
||||
|
||||
}
|
@ -30,7 +30,9 @@ import org.springframework.boot.context.properties.EnableConfigurationProperties
|
||||
import org.springframework.context.annotation.Bean
|
||||
import org.springframework.context.annotation.Configuration
|
||||
import org.springframework.core.annotation.Order
|
||||
import org.springframework.retry.support.RetryTemplate
|
||||
import org.springframework.web.client.RestTemplate
|
||||
import reactor.core.publisher.Sinks
|
||||
|
||||
@Configuration
|
||||
@EnableConfigurationProperties(
|
||||
@ -51,17 +53,22 @@ class AppRestConfiguration {
|
||||
}
|
||||
|
||||
@Bean
|
||||
fun restMtbFileSender(restTemplate: RestTemplate, restTargetProperties: RestTargetProperties): MtbFileSender {
|
||||
fun restMtbFileSender(
|
||||
restTemplate: RestTemplate,
|
||||
restTargetProperties: RestTargetProperties,
|
||||
retryTemplate: RetryTemplate
|
||||
): MtbFileSender {
|
||||
logger.info("Selected 'RestMtbFileSender'")
|
||||
return RestMtbFileSender(restTemplate, restTargetProperties)
|
||||
return RestMtbFileSender(restTemplate, restTargetProperties, retryTemplate)
|
||||
}
|
||||
|
||||
@Bean
|
||||
fun connectionCheckService(
|
||||
restTemplate: RestTemplate,
|
||||
restTargetProperties: RestTargetProperties
|
||||
restTargetProperties: RestTargetProperties,
|
||||
configsUpdateProducer: Sinks.Many<Boolean>
|
||||
): ConnectionCheckService {
|
||||
return RestConnectionCheckService(restTemplate, restTargetProperties)
|
||||
return RestConnectionCheckService(restTemplate, restTargetProperties, configsUpdateProducer)
|
||||
}
|
||||
|
||||
}
|
||||
|
@ -0,0 +1,129 @@
|
||||
/*
|
||||
* This file is part of ETL-Processor
|
||||
*
|
||||
* Copyright (c) 2023 Comprehensive Cancer Center Mainfranken, Datenintegrationszentrum Philipps-Universität Marburg and Contributors
|
||||
*
|
||||
* This program is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
* by the Free Software Foundation, either version 3 of the License, or
|
||||
* (at your option) any later version.
|
||||
*
|
||||
* This program is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
* GNU Affero General Public License for more details.
|
||||
*
|
||||
* You should have received a copy of the GNU Affero General Public License
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
*/
|
||||
|
||||
package dev.dnpm.etl.processor.config
|
||||
|
||||
import org.slf4j.LoggerFactory
|
||||
import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty
|
||||
import org.springframework.boot.context.properties.EnableConfigurationProperties
|
||||
import org.springframework.context.annotation.Bean
|
||||
import org.springframework.context.annotation.Configuration
|
||||
import org.springframework.security.config.annotation.web.builders.HttpSecurity
|
||||
import org.springframework.security.config.annotation.web.configuration.EnableWebSecurity
|
||||
import org.springframework.security.config.annotation.web.invoke
|
||||
import org.springframework.security.core.userdetails.User
|
||||
import org.springframework.security.core.userdetails.UserDetails
|
||||
import org.springframework.security.crypto.factory.PasswordEncoderFactories
|
||||
import org.springframework.security.crypto.password.PasswordEncoder
|
||||
import org.springframework.security.provisioning.InMemoryUserDetailsManager
|
||||
import org.springframework.security.web.SecurityFilterChain
|
||||
import java.util.*
|
||||
|
||||
|
||||
@Configuration
|
||||
@EnableConfigurationProperties(
|
||||
value = [
|
||||
SecurityConfigProperties::class
|
||||
]
|
||||
)
|
||||
@ConditionalOnProperty(value = ["app.security.admin-user"])
|
||||
@EnableWebSecurity
|
||||
class AppSecurityConfiguration(
|
||||
private val securityConfigProperties: SecurityConfigProperties
|
||||
) {
|
||||
|
||||
private val logger = LoggerFactory.getLogger(AppSecurityConfiguration::class.java)
|
||||
|
||||
@Bean
|
||||
fun userDetailsService(passwordEncoder: PasswordEncoder): InMemoryUserDetailsManager {
|
||||
val adminUser = if (securityConfigProperties.adminUser.isNullOrBlank()) {
|
||||
logger.warn("Using random Admin User: admin")
|
||||
"admin"
|
||||
} else {
|
||||
securityConfigProperties.adminUser
|
||||
}
|
||||
|
||||
val adminPassword = if (securityConfigProperties.adminPassword.isNullOrBlank()) {
|
||||
val random = UUID.randomUUID().toString()
|
||||
logger.warn("Using random Admin Passwort: {}", random)
|
||||
passwordEncoder.encode(random)
|
||||
} else {
|
||||
securityConfigProperties.adminPassword
|
||||
}
|
||||
|
||||
val user: UserDetails = User.withUsername(adminUser)
|
||||
.password(adminPassword)
|
||||
.roles("ADMIN")
|
||||
.build()
|
||||
|
||||
return InMemoryUserDetailsManager(user)
|
||||
}
|
||||
|
||||
@Bean
|
||||
@ConditionalOnProperty(value = ["app.security.enable-oidc"], havingValue = "true")
|
||||
fun filterChainOidc(http: HttpSecurity, passwordEncoder: PasswordEncoder): SecurityFilterChain {
|
||||
http {
|
||||
authorizeRequests {
|
||||
authorize("/configs/**", hasRole("ADMIN"))
|
||||
authorize("/mtbfile/**", hasAnyRole("MTBFILE"))
|
||||
authorize("/report/**", fullyAuthenticated)
|
||||
authorize(anyRequest, permitAll)
|
||||
}
|
||||
httpBasic {
|
||||
realmName = "ETL-Processor"
|
||||
}
|
||||
formLogin {
|
||||
loginPage = "/login"
|
||||
}
|
||||
oauth2Login {
|
||||
loginPage = "/login"
|
||||
}
|
||||
csrf { disable() }
|
||||
}
|
||||
return http.build()
|
||||
}
|
||||
|
||||
@Bean
|
||||
@ConditionalOnProperty(value = ["app.security.enable-oidc"], havingValue = "false", matchIfMissing = true)
|
||||
fun filterChain(http: HttpSecurity, passwordEncoder: PasswordEncoder): SecurityFilterChain {
|
||||
http {
|
||||
authorizeRequests {
|
||||
authorize("/configs/**", hasRole("ADMIN"))
|
||||
authorize("/mtbfile/**", hasAnyRole("MTBFILE"))
|
||||
authorize("/report/**", hasRole("ADMIN"))
|
||||
authorize(anyRequest, permitAll)
|
||||
}
|
||||
httpBasic {
|
||||
realmName = "ETL-Processor"
|
||||
}
|
||||
formLogin {
|
||||
loginPage = "/login"
|
||||
}
|
||||
csrf { disable() }
|
||||
}
|
||||
return http.build()
|
||||
}
|
||||
|
||||
@Bean
|
||||
fun passwordEncoder(): PasswordEncoder {
|
||||
return PasswordEncoderFactories.createDelegatingPasswordEncoder()
|
||||
}
|
||||
|
||||
}
|
||||
|
@ -24,9 +24,11 @@ import dev.dnpm.etl.processor.config.RestTargetProperties
|
||||
import jakarta.annotation.PostConstruct
|
||||
import org.apache.kafka.clients.consumer.Consumer
|
||||
import org.apache.kafka.common.errors.TimeoutException
|
||||
import org.springframework.beans.factory.annotation.Qualifier
|
||||
import org.springframework.http.HttpStatus
|
||||
import org.springframework.scheduling.annotation.Scheduled
|
||||
import org.springframework.web.client.RestTemplate
|
||||
import reactor.core.publisher.Sinks
|
||||
import kotlin.time.Duration.Companion.seconds
|
||||
import kotlin.time.toJavaDuration
|
||||
|
||||
@ -37,7 +39,9 @@ interface ConnectionCheckService {
|
||||
}
|
||||
|
||||
class KafkaConnectionCheckService(
|
||||
private val consumer: Consumer<String, String>
|
||||
private val consumer: Consumer<String, String>,
|
||||
@Qualifier("configsUpdateProducer")
|
||||
private val configsUpdateProducer: Sinks.Many<Boolean>
|
||||
) : ConnectionCheckService {
|
||||
|
||||
private var connectionAvailable: Boolean = false
|
||||
@ -51,6 +55,7 @@ class KafkaConnectionCheckService(
|
||||
} catch (e: TimeoutException) {
|
||||
false
|
||||
}
|
||||
configsUpdateProducer.emitNext(connectionAvailable, Sinks.EmitFailureHandler.FAIL_FAST)
|
||||
}
|
||||
|
||||
override fun connectionAvailable(): Boolean {
|
||||
@ -61,7 +66,9 @@ class KafkaConnectionCheckService(
|
||||
|
||||
class RestConnectionCheckService(
|
||||
private val restTemplate: RestTemplate,
|
||||
private val restTargetProperties: RestTargetProperties
|
||||
private val restTargetProperties: RestTargetProperties,
|
||||
@Qualifier("configsUpdateProducer")
|
||||
private val configsUpdateProducer: Sinks.Many<Boolean>
|
||||
) : ConnectionCheckService {
|
||||
|
||||
private var connectionAvailable: Boolean = false
|
||||
@ -77,6 +84,7 @@ class RestConnectionCheckService(
|
||||
} catch (e: Exception) {
|
||||
false
|
||||
}
|
||||
configsUpdateProducer.emitNext(connectionAvailable, Sinks.EmitFailureHandler.FAIL_FAST)
|
||||
}
|
||||
|
||||
override fun connectionAvailable(): Boolean {
|
||||
|
@ -20,10 +20,13 @@
|
||||
package dev.dnpm.etl.processor.monitoring
|
||||
|
||||
import org.springframework.data.annotation.Id
|
||||
import org.springframework.data.domain.Page
|
||||
import org.springframework.data.domain.Pageable
|
||||
import org.springframework.data.jdbc.repository.query.Query
|
||||
import org.springframework.data.relational.core.mapping.Embedded
|
||||
import org.springframework.data.relational.core.mapping.Table
|
||||
import org.springframework.data.repository.CrudRepository
|
||||
import org.springframework.data.repository.PagingAndSortingRepository
|
||||
import java.time.Instant
|
||||
import java.util.*
|
||||
|
||||
@ -52,12 +55,14 @@ data class CountedState(
|
||||
val status: RequestStatus,
|
||||
)
|
||||
|
||||
interface RequestRepository : CrudRepository<Request, Long> {
|
||||
interface RequestRepository : CrudRepository<Request, Long>, PagingAndSortingRepository<Request, Long> {
|
||||
|
||||
fun findAllByPatientIdOrderByProcessedAtDesc(patientId: String): List<Request>
|
||||
|
||||
fun findByUuidEquals(uuid: String): Optional<Request>
|
||||
|
||||
fun findRequestByPatientId(patientId: String, pageable: Pageable): Page<Request>
|
||||
|
||||
@Query("SELECT count(*) AS count, status FROM request WHERE type = 'MTB_FILE' GROUP BY status ORDER BY status, count DESC;")
|
||||
fun countStates(): List<CountedState>
|
||||
|
||||
|
@ -1,7 +1,7 @@
|
||||
/*
|
||||
* This file is part of ETL-Processor
|
||||
*
|
||||
* Copyright (c) 2023 Comprehensive Cancer Center Mainfranken, Datenintegrationszentrum Philipps-Universität Marburg and Contributors
|
||||
* Copyright (c) 2024 Comprehensive Cancer Center Mainfranken, Datenintegrationszentrum Philipps-Universität Marburg and Contributors
|
||||
*
|
||||
* This program is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
@ -26,10 +26,12 @@ import dev.dnpm.etl.processor.config.KafkaTargetProperties
|
||||
import dev.dnpm.etl.processor.monitoring.RequestStatus
|
||||
import org.slf4j.LoggerFactory
|
||||
import org.springframework.kafka.core.KafkaTemplate
|
||||
import org.springframework.retry.support.RetryTemplate
|
||||
|
||||
class KafkaMtbFileSender(
|
||||
private val kafkaTemplate: KafkaTemplate<String, String>,
|
||||
private val kafkaTargetProperties: KafkaTargetProperties,
|
||||
private val retryTemplate: RetryTemplate,
|
||||
private val objectMapper: ObjectMapper
|
||||
) : MtbFileSender {
|
||||
|
||||
@ -37,16 +39,18 @@ class KafkaMtbFileSender(
|
||||
|
||||
override fun send(request: MtbFileSender.MtbFileRequest): MtbFileSender.Response {
|
||||
return try {
|
||||
val result = kafkaTemplate.send(
|
||||
kafkaTargetProperties.topic,
|
||||
key(request),
|
||||
objectMapper.writeValueAsString(Data(request.requestId, request.mtbFile))
|
||||
)
|
||||
if (result.get() != null) {
|
||||
logger.debug("Sent file via KafkaMtbFileSender")
|
||||
MtbFileSender.Response(RequestStatus.UNKNOWN)
|
||||
} else {
|
||||
MtbFileSender.Response(RequestStatus.ERROR)
|
||||
return retryTemplate.execute<MtbFileSender.Response, Exception> {
|
||||
val result = kafkaTemplate.send(
|
||||
kafkaTargetProperties.topic,
|
||||
key(request),
|
||||
objectMapper.writeValueAsString(Data(request.requestId, request.mtbFile))
|
||||
)
|
||||
if (result.get() != null) {
|
||||
logger.debug("Sent file via KafkaMtbFileSender")
|
||||
MtbFileSender.Response(RequestStatus.UNKNOWN)
|
||||
} else {
|
||||
MtbFileSender.Response(RequestStatus.ERROR)
|
||||
}
|
||||
}
|
||||
} catch (e: Exception) {
|
||||
logger.error("An error occurred sending to kafka", e)
|
||||
@ -65,17 +69,19 @@ class KafkaMtbFileSender(
|
||||
.build()
|
||||
|
||||
return try {
|
||||
val result = kafkaTemplate.send(
|
||||
kafkaTargetProperties.topic,
|
||||
key(request),
|
||||
objectMapper.writeValueAsString(Data(request.requestId, dummyMtbFile))
|
||||
)
|
||||
return retryTemplate.execute<MtbFileSender.Response, Exception> {
|
||||
val result = kafkaTemplate.send(
|
||||
kafkaTargetProperties.topic,
|
||||
key(request),
|
||||
objectMapper.writeValueAsString(Data(request.requestId, dummyMtbFile))
|
||||
)
|
||||
|
||||
if (result.get() != null) {
|
||||
logger.debug("Sent deletion request via KafkaMtbFileSender")
|
||||
MtbFileSender.Response(RequestStatus.UNKNOWN)
|
||||
} else {
|
||||
MtbFileSender.Response(RequestStatus.ERROR)
|
||||
if (result.get() != null) {
|
||||
logger.debug("Sent deletion request via KafkaMtbFileSender")
|
||||
MtbFileSender.Response(RequestStatus.UNKNOWN)
|
||||
} else {
|
||||
MtbFileSender.Response(RequestStatus.ERROR)
|
||||
}
|
||||
}
|
||||
} catch (e: Exception) {
|
||||
logger.error("An error occurred sending to kafka", e)
|
||||
@ -83,6 +89,10 @@ class KafkaMtbFileSender(
|
||||
}
|
||||
}
|
||||
|
||||
override fun endpoint(): String {
|
||||
return "${this.kafkaTargetProperties.servers} (${this.kafkaTargetProperties.topic}/${this.kafkaTargetProperties.responseTopic})"
|
||||
}
|
||||
|
||||
private fun key(request: MtbFileSender.MtbFileRequest): String {
|
||||
return "{\"pid\": \"${request.mtbFile.patient.id}\", " +
|
||||
"\"eid\": \"${request.mtbFile.episode.id}\"}"
|
||||
|
@ -28,6 +28,8 @@ interface MtbFileSender {
|
||||
|
||||
fun send(request: DeleteRequest): Response
|
||||
|
||||
fun endpoint(): String
|
||||
|
||||
data class Response(val status: RequestStatus, val body: String = "")
|
||||
|
||||
data class MtbFileRequest(val requestId: String, val mtbFile: MtbFile)
|
||||
|
@ -1,7 +1,7 @@
|
||||
/*
|
||||
* This file is part of ETL-Processor
|
||||
*
|
||||
* Copyright (c) 2023 Comprehensive Cancer Center Mainfranken, Datenintegrationszentrum Philipps-Universität Marburg and Contributors
|
||||
* Copyright (c) 2024 Comprehensive Cancer Center Mainfranken, Datenintegrationszentrum Philipps-Universität Marburg and Contributors
|
||||
*
|
||||
* This program is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
@ -25,32 +25,39 @@ import org.slf4j.LoggerFactory
|
||||
import org.springframework.http.HttpEntity
|
||||
import org.springframework.http.HttpHeaders
|
||||
import org.springframework.http.MediaType
|
||||
import org.springframework.retry.support.RetryTemplate
|
||||
import org.springframework.web.client.RestClientException
|
||||
import org.springframework.web.client.RestTemplate
|
||||
|
||||
class RestMtbFileSender(
|
||||
private val restTemplate: RestTemplate,
|
||||
private val restTargetProperties: RestTargetProperties
|
||||
private val restTargetProperties: RestTargetProperties,
|
||||
private val retryTemplate: RetryTemplate
|
||||
) : MtbFileSender {
|
||||
|
||||
private val logger = LoggerFactory.getLogger(RestMtbFileSender::class.java)
|
||||
|
||||
override fun send(request: MtbFileSender.MtbFileRequest): MtbFileSender.Response {
|
||||
try {
|
||||
val headers = HttpHeaders()
|
||||
headers.contentType = MediaType.APPLICATION_JSON
|
||||
val entityReq = HttpEntity(request.mtbFile, headers)
|
||||
val response = restTemplate.postForEntity(
|
||||
"${restTargetProperties.uri}/MTBFile",
|
||||
entityReq,
|
||||
String::class.java
|
||||
)
|
||||
if (!response.statusCode.is2xxSuccessful) {
|
||||
logger.warn("Error sending to remote system: {}", response.body)
|
||||
return MtbFileSender.Response(response.statusCode.asRequestStatus(), "Status-Code: ${response.statusCode.value()}")
|
||||
return retryTemplate.execute<MtbFileSender.Response, Exception> {
|
||||
val headers = HttpHeaders()
|
||||
headers.contentType = MediaType.APPLICATION_JSON
|
||||
val entityReq = HttpEntity(request.mtbFile, headers)
|
||||
val response = restTemplate.postForEntity(
|
||||
"${restTargetProperties.uri}/MTBFile",
|
||||
entityReq,
|
||||
String::class.java
|
||||
)
|
||||
if (!response.statusCode.is2xxSuccessful) {
|
||||
logger.warn("Error sending to remote system: {}", response.body)
|
||||
return@execute MtbFileSender.Response(
|
||||
response.statusCode.asRequestStatus(),
|
||||
"Status-Code: ${response.statusCode.value()}"
|
||||
)
|
||||
}
|
||||
logger.debug("Sent file via RestMtbFileSender")
|
||||
return@execute MtbFileSender.Response(response.statusCode.asRequestStatus(), response.body.orEmpty())
|
||||
}
|
||||
logger.debug("Sent file via RestMtbFileSender")
|
||||
return MtbFileSender.Response(response.statusCode.asRequestStatus(), response.body.orEmpty())
|
||||
} catch (e: IllegalArgumentException) {
|
||||
logger.error("Not a valid URI to export to: '{}'", restTargetProperties.uri!!)
|
||||
} catch (e: RestClientException) {
|
||||
@ -62,16 +69,18 @@ class RestMtbFileSender(
|
||||
|
||||
override fun send(request: MtbFileSender.DeleteRequest): MtbFileSender.Response {
|
||||
try {
|
||||
val headers = HttpHeaders()
|
||||
headers.contentType = MediaType.APPLICATION_JSON
|
||||
val entityReq = HttpEntity(null, headers)
|
||||
restTemplate.delete(
|
||||
"${restTargetProperties.uri}/Patient/${request.patientId}",
|
||||
entityReq,
|
||||
String::class.java
|
||||
)
|
||||
logger.debug("Sent file via RestMtbFileSender")
|
||||
return MtbFileSender.Response(RequestStatus.SUCCESS)
|
||||
return retryTemplate.execute<MtbFileSender.Response, Exception> {
|
||||
val headers = HttpHeaders()
|
||||
headers.contentType = MediaType.APPLICATION_JSON
|
||||
val entityReq = HttpEntity(null, headers)
|
||||
restTemplate.delete(
|
||||
"${restTargetProperties.uri}/Patient/${request.patientId}",
|
||||
entityReq,
|
||||
String::class.java
|
||||
)
|
||||
logger.debug("Sent file via RestMtbFileSender")
|
||||
return@execute MtbFileSender.Response(RequestStatus.SUCCESS)
|
||||
}
|
||||
} catch (e: IllegalArgumentException) {
|
||||
logger.error("Not a valid URI to export to: '{}'", restTargetProperties.uri!!)
|
||||
} catch (e: RestClientException) {
|
||||
@ -81,4 +90,8 @@ class RestMtbFileSender(
|
||||
return MtbFileSender.Response(RequestStatus.ERROR, "Sonstiger Fehler bei der Übertragung")
|
||||
}
|
||||
|
||||
override fun endpoint(): String {
|
||||
return this.restTargetProperties.uri.orEmpty()
|
||||
}
|
||||
|
||||
}
|
@ -0,0 +1,92 @@
|
||||
/*
|
||||
* This file is part of ETL-Processor
|
||||
*
|
||||
* Copyright (c) 2024 Comprehensive Cancer Center Mainfranken, Datenintegrationszentrum Philipps-Universität Marburg and Contributors
|
||||
*
|
||||
* This program is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
* by the Free Software Foundation, either version 3 of the License, or
|
||||
* (at your option) any later version.
|
||||
*
|
||||
* This program is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
* GNU Affero General Public License for more details.
|
||||
*
|
||||
* You should have received a copy of the GNU Affero General Public License
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
*/
|
||||
|
||||
package dev.dnpm.etl.processor.services
|
||||
|
||||
import jakarta.annotation.PostConstruct
|
||||
import org.springframework.data.annotation.Id
|
||||
import org.springframework.data.relational.core.mapping.Table
|
||||
import org.springframework.data.repository.CrudRepository
|
||||
import org.springframework.data.repository.findByIdOrNull
|
||||
import org.springframework.security.core.userdetails.User
|
||||
import org.springframework.security.crypto.password.PasswordEncoder
|
||||
import org.springframework.security.provisioning.InMemoryUserDetailsManager
|
||||
import java.time.Instant
|
||||
import java.util.*
|
||||
|
||||
class TokenService(
|
||||
private val userDetailsManager: InMemoryUserDetailsManager,
|
||||
private val passwordEncoder: PasswordEncoder,
|
||||
private val tokenRepository: TokenRepository
|
||||
) {
|
||||
|
||||
@PostConstruct
|
||||
fun setup() {
|
||||
tokenRepository.findAll().forEach {
|
||||
userDetailsManager.createUser(
|
||||
User.withUsername(it.username)
|
||||
.password(it.password)
|
||||
.roles("MTBFILE")
|
||||
.build()
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
fun addToken(name: String): Result<String> {
|
||||
val username = name.lowercase().replace("""[^a-z0-9]""".toRegex(), "")
|
||||
if (userDetailsManager.userExists(username)) {
|
||||
return Result.failure(RuntimeException("Cannot use token name"))
|
||||
}
|
||||
|
||||
val password = Base64.getEncoder().encodeToString(UUID.randomUUID().toString().encodeToByteArray())
|
||||
val encodedPassword = passwordEncoder.encode(password).toString()
|
||||
|
||||
userDetailsManager.createUser(
|
||||
User.withUsername(username)
|
||||
.password(encodedPassword)
|
||||
.roles("MTBFILE")
|
||||
.build()
|
||||
)
|
||||
|
||||
tokenRepository.save(Token(name = name, username = username, password = encodedPassword))
|
||||
|
||||
return Result.success("$username:$password")
|
||||
}
|
||||
|
||||
fun deleteToken(id: Long) {
|
||||
val token = tokenRepository.findByIdOrNull(id) ?: return
|
||||
userDetailsManager.deleteUser(token.username)
|
||||
tokenRepository.delete(token)
|
||||
}
|
||||
|
||||
fun findAll(): List<Token> {
|
||||
return tokenRepository.findAll().toList()
|
||||
}
|
||||
}
|
||||
|
||||
@Table("token")
|
||||
data class Token(
|
||||
@Id val id: Long? = null,
|
||||
val name: String,
|
||||
val username: String,
|
||||
val password: String,
|
||||
val createdAt: Instant = Instant.now()
|
||||
)
|
||||
|
||||
interface TokenRepository : CrudRepository<Token, Long>
|
@ -22,30 +22,103 @@ package dev.dnpm.etl.processor.web
|
||||
import dev.dnpm.etl.processor.monitoring.ConnectionCheckService
|
||||
import dev.dnpm.etl.processor.output.MtbFileSender
|
||||
import dev.dnpm.etl.processor.pseudonym.Generator
|
||||
import dev.dnpm.etl.processor.services.Token
|
||||
import dev.dnpm.etl.processor.services.TokenService
|
||||
import dev.dnpm.etl.processor.services.TransformationService
|
||||
import org.springframework.beans.factory.annotation.Qualifier
|
||||
import org.springframework.http.MediaType
|
||||
import org.springframework.http.codec.ServerSentEvent
|
||||
import org.springframework.stereotype.Controller
|
||||
import org.springframework.ui.Model
|
||||
import org.springframework.web.bind.annotation.GetMapping
|
||||
import org.springframework.web.bind.annotation.RequestMapping
|
||||
import org.springframework.web.bind.annotation.*
|
||||
import reactor.core.publisher.Flux
|
||||
import reactor.core.publisher.Sinks
|
||||
|
||||
@Controller
|
||||
@RequestMapping(path = ["configs"])
|
||||
class ConfigController(
|
||||
@Qualifier("configsUpdateProducer")
|
||||
private val configsUpdateProducer: Sinks.Many<Boolean>,
|
||||
private val transformationService: TransformationService,
|
||||
private val pseudonymGenerator: Generator,
|
||||
private val mtbFileSender: MtbFileSender,
|
||||
private val connectionCheckService: ConnectionCheckService
|
||||
|
||||
private val connectionCheckService: ConnectionCheckService,
|
||||
private val tokenService: TokenService?
|
||||
) {
|
||||
|
||||
@GetMapping
|
||||
fun index(model: Model): String {
|
||||
model.addAttribute("pseudonymGenerator", pseudonymGenerator.javaClass.simpleName)
|
||||
model.addAttribute("mtbFileSender", mtbFileSender.javaClass.simpleName)
|
||||
model.addAttribute("mtbFileEndpoint", mtbFileSender.endpoint())
|
||||
model.addAttribute("connectionAvailable", connectionCheckService.connectionAvailable())
|
||||
model.addAttribute("tokensEnabled", tokenService != null)
|
||||
if (tokenService != null) {
|
||||
model.addAttribute("tokens", tokenService.findAll())
|
||||
} else {
|
||||
model.addAttribute("tokens", listOf<Token>())
|
||||
}
|
||||
model.addAttribute("transformations", transformationService.getTransformations())
|
||||
|
||||
return "configs"
|
||||
}
|
||||
|
||||
@GetMapping(params = ["connectionAvailable"])
|
||||
fun connectionAvailable(model: Model): String {
|
||||
model.addAttribute("mtbFileSender", mtbFileSender.javaClass.simpleName)
|
||||
model.addAttribute("mtbFileEndpoint", mtbFileSender.endpoint())
|
||||
model.addAttribute("connectionAvailable", connectionCheckService.connectionAvailable())
|
||||
if (tokenService != null) {
|
||||
model.addAttribute("tokensEnabled", true)
|
||||
model.addAttribute("tokens", tokenService.findAll())
|
||||
} else {
|
||||
model.addAttribute("tokens", listOf<Token>())
|
||||
}
|
||||
|
||||
return "configs/connectionAvailable"
|
||||
}
|
||||
|
||||
@PostMapping(path = ["tokens"])
|
||||
fun addToken(@ModelAttribute("name") name: String, model: Model): String {
|
||||
if (tokenService == null) {
|
||||
model.addAttribute("tokensEnabled", false)
|
||||
model.addAttribute("success", false)
|
||||
} else {
|
||||
model.addAttribute("tokensEnabled", true)
|
||||
val result = tokenService.addToken(name)
|
||||
if (result.isSuccess) {
|
||||
model.addAttribute("newTokenValue", result.getOrDefault(""))
|
||||
model.addAttribute("success", true)
|
||||
} else {
|
||||
model.addAttribute("success", false)
|
||||
}
|
||||
model.addAttribute("tokens", tokenService.findAll())
|
||||
}
|
||||
|
||||
return "configs/tokens"
|
||||
}
|
||||
|
||||
@DeleteMapping(path = ["tokens/{id}"])
|
||||
fun deleteToken(@PathVariable id: Long, model: Model): String {
|
||||
if (tokenService != null) {
|
||||
tokenService.deleteToken(id)
|
||||
|
||||
model.addAttribute("tokensEnabled", true)
|
||||
model.addAttribute("tokens", tokenService.findAll())
|
||||
} else {
|
||||
model.addAttribute("tokensEnabled", false)
|
||||
model.addAttribute("tokens", listOf<Token>())
|
||||
}
|
||||
return "configs/tokens"
|
||||
}
|
||||
|
||||
@GetMapping(path = ["events"], produces = [MediaType.TEXT_EVENT_STREAM_VALUE])
|
||||
fun events(): Flux<ServerSentEvent<Any>> {
|
||||
return configsUpdateProducer.asFlux().map {
|
||||
ServerSentEvent.builder<Any>()
|
||||
.event("connection-available").id("none").data("")
|
||||
.build()
|
||||
}
|
||||
}
|
||||
|
||||
}
|
@ -23,6 +23,9 @@ import dev.dnpm.etl.processor.NotFoundException
|
||||
import dev.dnpm.etl.processor.monitoring.ReportService
|
||||
import dev.dnpm.etl.processor.monitoring.RequestId
|
||||
import dev.dnpm.etl.processor.monitoring.RequestRepository
|
||||
import org.springframework.data.domain.Pageable
|
||||
import org.springframework.data.domain.Sort
|
||||
import org.springframework.data.web.PageableDefault
|
||||
import org.springframework.stereotype.Controller
|
||||
import org.springframework.ui.Model
|
||||
import org.springframework.web.bind.annotation.GetMapping
|
||||
@ -37,8 +40,24 @@ class HomeController(
|
||||
) {
|
||||
|
||||
@GetMapping
|
||||
fun index(model: Model): String {
|
||||
val requests = requestRepository.findAll().sortedByDescending { it.processedAt }.take(25)
|
||||
fun index(
|
||||
@PageableDefault(page = 0, size = 20, sort = ["processedAt"], direction = Sort.Direction.DESC) pageable: Pageable,
|
||||
model: Model
|
||||
): String {
|
||||
val requests = requestRepository.findAll(pageable)
|
||||
model.addAttribute("requests", requests)
|
||||
|
||||
return "index"
|
||||
}
|
||||
|
||||
@GetMapping(path = ["patient/{patientId}"])
|
||||
fun byPatient(
|
||||
@PathVariable patientId: String,
|
||||
@PageableDefault(page = 0, size = 20, sort = ["processedAt"], direction = Sort.Direction.DESC) pageable: Pageable,
|
||||
model: Model
|
||||
): String {
|
||||
val requests = requestRepository.findRequestByPatientId(patientId, pageable)
|
||||
model.addAttribute("patientId", patientId)
|
||||
model.addAttribute("requests", requests)
|
||||
|
||||
return "index"
|
||||
|
@ -0,0 +1,47 @@
|
||||
/*
|
||||
* This file is part of ETL-Processor
|
||||
*
|
||||
* Copyright (c) 2024 Comprehensive Cancer Center Mainfranken, Datenintegrationszentrum Philipps-Universität Marburg and Contributors
|
||||
*
|
||||
* This program is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
* by the Free Software Foundation, either version 3 of the License, or
|
||||
* (at your option) any later version.
|
||||
*
|
||||
* This program is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
* GNU Affero General Public License for more details.
|
||||
*
|
||||
* You should have received a copy of the GNU Affero General Public License
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
*/
|
||||
|
||||
package dev.dnpm.etl.processor.web
|
||||
|
||||
import dev.dnpm.etl.processor.config.SecurityConfigProperties
|
||||
import org.springframework.boot.autoconfigure.security.oauth2.client.OAuth2ClientProperties
|
||||
import org.springframework.stereotype.Controller
|
||||
import org.springframework.ui.Model
|
||||
import org.springframework.web.bind.annotation.GetMapping
|
||||
|
||||
@Controller
|
||||
class LoginController(
|
||||
private val securityConfigProperties: SecurityConfigProperties?,
|
||||
private val oAuth2ClientProperties: OAuth2ClientProperties?
|
||||
) {
|
||||
|
||||
@GetMapping(path = ["/login"])
|
||||
fun login(model: Model): String {
|
||||
if (securityConfigProperties?.enableOidc == true) {
|
||||
model.addAttribute(
|
||||
"oidcLogins",
|
||||
oAuth2ClientProperties?.registration?.map { (key, value) -> Pair(key, value.clientName) }.orEmpty()
|
||||
)
|
||||
} else {
|
||||
model.addAttribute("oidcLogins", emptyList<Pair<String, String>>())
|
||||
}
|
||||
return "login"
|
||||
}
|
||||
|
||||
}
|
@ -27,13 +27,19 @@ import org.springframework.http.ResponseEntity
|
||||
import org.springframework.web.bind.annotation.*
|
||||
|
||||
@RestController
|
||||
@RequestMapping(path = ["mtbfile"])
|
||||
class MtbFileRestController(
|
||||
private val requestProcessor: RequestProcessor,
|
||||
) {
|
||||
|
||||
private val logger = LoggerFactory.getLogger(MtbFileRestController::class.java)
|
||||
|
||||
@PostMapping(path = ["/mtbfile"])
|
||||
@GetMapping
|
||||
fun info(): ResponseEntity<String> {
|
||||
return ResponseEntity.ok("Test")
|
||||
}
|
||||
|
||||
@PostMapping
|
||||
fun mtbFile(@RequestBody mtbFile: MtbFile): ResponseEntity<Void> {
|
||||
if (mtbFile.consent.status == Consent.Status.ACTIVE) {
|
||||
logger.debug("Accepted MTB File for processing")
|
||||
@ -45,7 +51,7 @@ class MtbFileRestController(
|
||||
return ResponseEntity.accepted().build()
|
||||
}
|
||||
|
||||
@DeleteMapping(path = ["/mtbfile/{patientId}"])
|
||||
@DeleteMapping(path = ["{patientId}"])
|
||||
fun deleteData(@PathVariable patientId: String): ResponseEntity<Void> {
|
||||
logger.debug("Accepted patient ID to process deletion")
|
||||
requestProcessor.processDeletion(patientId)
|
||||
|
@ -22,6 +22,7 @@ package dev.dnpm.etl.processor.web
|
||||
import dev.dnpm.etl.processor.monitoring.RequestRepository
|
||||
import dev.dnpm.etl.processor.monitoring.RequestStatus
|
||||
import dev.dnpm.etl.processor.monitoring.RequestType
|
||||
import org.springframework.beans.factory.annotation.Qualifier
|
||||
import org.springframework.http.MediaType
|
||||
import org.springframework.http.codec.ServerSentEvent
|
||||
import org.springframework.web.bind.annotation.GetMapping
|
||||
@ -38,6 +39,7 @@ import java.time.temporal.ChronoUnit
|
||||
@RestController
|
||||
@RequestMapping(path = ["/statistics"])
|
||||
class StatisticsRestController(
|
||||
@Qualifier("statisticsUpdateProducer")
|
||||
private val statisticsUpdateProducer: Sinks.Many<Any>,
|
||||
private val requestRepository: RequestRepository
|
||||
) {
|
||||
@ -132,6 +134,7 @@ class StatisticsRestController(
|
||||
@GetMapping(path = ["events"], produces = [MediaType.TEXT_EVENT_STREAM_VALUE])
|
||||
fun updater(): Flux<ServerSentEvent<Any>> {
|
||||
return statisticsUpdateProducer.asFlux().flatMap {
|
||||
println(it)
|
||||
Flux.fromIterable(
|
||||
listOf(
|
||||
ServerSentEvent.builder<Any>()
|
||||
@ -152,6 +155,10 @@ class StatisticsRestController(
|
||||
.build(),
|
||||
ServerSentEvent.builder<Any>()
|
||||
.event("deleterequestpatientstates").id("none").data(this.requestPatientStates(true))
|
||||
.build(),
|
||||
|
||||
ServerSentEvent.builder<Any>()
|
||||
.event("newrequest").id("none").data("newrequest")
|
||||
.build()
|
||||
)
|
||||
)
|
||||
|
@ -10,6 +10,9 @@ app:
|
||||
topic: test
|
||||
response-topic: test_response
|
||||
servers: localhost:9094
|
||||
#security:
|
||||
# admin-user: admin
|
||||
# admin-password: "{noop}very-secret"
|
||||
|
||||
server:
|
||||
port: 8000
|
||||
|
@ -4,4 +4,7 @@ spring:
|
||||
consumer:
|
||||
group-id: ${app.kafka.group-id}
|
||||
flyway:
|
||||
locations: "classpath:db/migration/{vendor}"
|
||||
locations: "classpath:db/migration/{vendor}"
|
||||
|
||||
server:
|
||||
forward-headers-strategy: framework
|
@ -0,0 +1,8 @@
|
||||
CREATE TABLE IF NOT EXISTS token
|
||||
(
|
||||
id int auto_increment primary key,
|
||||
name varchar(255) not null,
|
||||
username varchar(255) not null unique,
|
||||
password varchar(255) not null,
|
||||
created_at datetime default utc_timestamp() not null
|
||||
);
|
@ -0,0 +1,9 @@
|
||||
CREATE TABLE IF NOT EXISTS token
|
||||
(
|
||||
id serial,
|
||||
name varchar(255) not null,
|
||||
username varchar(255) not null unique,
|
||||
password varchar(255) not null,
|
||||
created_at timestamp with time zone default now() not null,
|
||||
PRIMARY KEY (id)
|
||||
);
|
BIN
src/main/resources/static/bg.jpeg
Normal file
BIN
src/main/resources/static/bg.jpeg
Normal file
Binary file not shown.
After Width: | Height: | Size: 3.0 KiB |
45
src/main/resources/static/echarts.min.js
vendored
45
src/main/resources/static/echarts.min.js
vendored
File diff suppressed because one or more lines are too long
41
src/main/resources/static/icon.svg
Normal file
41
src/main/resources/static/icon.svg
Normal file
@ -0,0 +1,41 @@
|
||||
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
|
||||
<!-- Created with Inkscape (http://www.inkscape.org/) -->
|
||||
|
||||
<svg
|
||||
width="256"
|
||||
height="256"
|
||||
viewBox="0 0 67.733332 67.733335"
|
||||
version="1.1"
|
||||
id="svg5"
|
||||
xmlns="http://www.w3.org/2000/svg"
|
||||
xmlns:svg="http://www.w3.org/2000/svg">
|
||||
<defs
|
||||
id="defs2" />
|
||||
<g
|
||||
id="layer1">
|
||||
<g
|
||||
id="g26002"
|
||||
transform="matrix(1.5,0,0,1.5,-16.933333,-1.8487648)">
|
||||
<path
|
||||
id="path12437"
|
||||
transform="matrix(0.21771408,0,0,0.21771408,73.025692,24.874779)"
|
||||
style="fill:#f59e00;fill-opacity:1"
|
||||
d="m -110.41995,43.223174 -55.55561,-2e-6 -27.7778,-48.1125685 27.77781,-48.1125655 55.5556,3e-6 27.777803,48.1125679 z" />
|
||||
<path
|
||||
id="path13446"
|
||||
transform="matrix(0.21771408,0,0,0.21771408,54.882836,14.399994)"
|
||||
style="fill:#004d6e;fill-opacity:1"
|
||||
d="m -110.41995,43.223174 -55.55561,-2e-6 -27.7778,-48.1125685 27.77781,-48.1125655 55.5556,3e-6 27.777803,48.1125679 z" />
|
||||
<path
|
||||
id="path13448"
|
||||
transform="matrix(0.21771408,0,0,0.21771408,54.882835,35.349561)"
|
||||
style="fill:#706f6f;fill-opacity:1"
|
||||
d="m -110.41995,43.223174 -55.55561,-2e-6 -27.7778,-48.1125685 27.77781,-48.1125655 55.5556,3e-6 27.777803,48.1125679 z" />
|
||||
<path
|
||||
id="path25844"
|
||||
transform="matrix(0.21771408,0,0,0.21771408,60.930454,24.874778)"
|
||||
style="fill:#ffffff;fill-opacity:1"
|
||||
d="m -110.41995,43.223174 -55.55561,-2e-6 -27.7778,-48.1125685 27.77781,-48.1125655 55.5556,3e-6 27.777803,48.1125679 z" />
|
||||
</g>
|
||||
</g>
|
||||
</svg>
|
After Width: | Height: | Size: 1.6 KiB |
BIN
src/main/resources/static/kafka.png
Normal file
BIN
src/main/resources/static/kafka.png
Normal file
Binary file not shown.
After Width: | Height: | Size: 1.6 KiB |
@ -4,7 +4,7 @@ const dateFormat = new Intl.DateTimeFormat('de-DE', dateFormatOptions);
|
||||
const dateTimeFormatOptions = { year: 'numeric', month: '2-digit', day: '2-digit', hour: '2-digit', minute: 'numeric', second: 'numeric' };
|
||||
const dateTimeFormat = new Intl.DateTimeFormat('de-DE', dateTimeFormatOptions);
|
||||
|
||||
window.onload = () => {
|
||||
const formatTimeElements = () => {
|
||||
Array.from(document.getElementsByTagName('time')).forEach((timeTag) => {
|
||||
let date = Date.parse(timeTag.getAttribute('datetime'));
|
||||
if (! isNaN(date)) {
|
||||
@ -13,6 +13,9 @@ window.onload = () => {
|
||||
});
|
||||
};
|
||||
|
||||
window.addEventListener('load', formatTimeElements);
|
||||
window.addEventListener('htmx:afterRequest', formatTimeElements);
|
||||
|
||||
function drawPieChart(url, elemId, title, data) {
|
||||
if (data) {
|
||||
update(elemId, data);
|
||||
|
BIN
src/main/resources/static/server.png
Normal file
BIN
src/main/resources/static/server.png
Normal file
Binary file not shown.
After Width: | Height: | Size: 1.4 KiB |
@ -1,5 +1,6 @@
|
||||
:root {
|
||||
--table-border: rgba(96, 96, 96, 1);
|
||||
--text: #333;
|
||||
--table-border: rgba(16, 24, 40, .1);
|
||||
|
||||
--bg-blue: rgb(0, 74, 157);
|
||||
--bg-blue-op: rgba(0, 74, 157, .35);
|
||||
@ -19,47 +20,85 @@
|
||||
--bg-gray-op: rgba(112, 128, 144, .35);
|
||||
}
|
||||
|
||||
html {
|
||||
background: linear-gradient(-5deg, var(--bg-blue-op), transparent 10em);
|
||||
min-height: 100vh;
|
||||
overflow-y: scroll;
|
||||
}
|
||||
|
||||
body {
|
||||
margin: 0;
|
||||
margin: 0 0 5em 0;
|
||||
font-family: sans-serif;
|
||||
font-size: .8rem;
|
||||
color: #333;
|
||||
color: var(--text);
|
||||
|
||||
min-height: 100vh;
|
||||
|
||||
background: url(bg.jpeg) no-repeat;
|
||||
background-size: contain;
|
||||
}
|
||||
|
||||
nav {
|
||||
margin: 0 auto;
|
||||
background: #d5dad5;
|
||||
height: 3rem;
|
||||
padding: 2em 0;
|
||||
|
||||
line-height: 1.5rem;
|
||||
max-width: 1140px;
|
||||
|
||||
border-bottom: 1px solid var(--table-border);
|
||||
}
|
||||
|
||||
nav a {
|
||||
color: #004a8f;
|
||||
text-transform: uppercase;
|
||||
nav > a.nav-home {
|
||||
float: left;
|
||||
|
||||
color: var(--text);
|
||||
line-height: 1.5em;
|
||||
text-decoration: none;
|
||||
line-height: 2rem;
|
||||
font-weight: 700;
|
||||
|
||||
font-size: 1.5em;
|
||||
font-weight: bold;
|
||||
}
|
||||
|
||||
nav a:hover {
|
||||
text-decoration: underline;
|
||||
nav > a.nav-home > img {
|
||||
width: 1.5em;
|
||||
vertical-align: middle;
|
||||
}
|
||||
|
||||
nav > ul {
|
||||
margin: 0 3rem;
|
||||
margin: 0 0 0 auto;
|
||||
padding: 0;
|
||||
|
||||
width: max-content;
|
||||
}
|
||||
|
||||
nav > ul > li {
|
||||
background: #fbfbfb;
|
||||
display: block;
|
||||
float: left;
|
||||
padding: 2px 1rem;
|
||||
border-left: 1px solid #d5dad5;
|
||||
display: inline-block;
|
||||
padding: 0 1rem;
|
||||
}
|
||||
|
||||
nav > ul > li:first-of-type {
|
||||
border-left: none;
|
||||
nav > ul > li.login {
|
||||
margin: 0 0 0 1em;
|
||||
padding: 0 0 0 2em;
|
||||
border-left: 1px solid var(--table-border);
|
||||
}
|
||||
|
||||
nav li a {
|
||||
color: var(--bg-blue);
|
||||
text-transform: uppercase;
|
||||
text-decoration: none;
|
||||
font-weight: 700;
|
||||
}
|
||||
|
||||
nav li.login a {
|
||||
color: var(--bg-red);
|
||||
}
|
||||
|
||||
nav li a:hover {
|
||||
text-decoration: underline;
|
||||
}
|
||||
|
||||
a {
|
||||
color: var(--bg-blue);
|
||||
}
|
||||
|
||||
.breadcrumps {
|
||||
@ -85,7 +124,7 @@ nav > ul > li:first-of-type {
|
||||
}
|
||||
|
||||
.breadcrumps ul li a {
|
||||
color: #333333;
|
||||
color: var(--text);
|
||||
text-decoration: none;
|
||||
}
|
||||
|
||||
@ -98,6 +137,10 @@ main {
|
||||
max-width: 1140px;
|
||||
}
|
||||
|
||||
section {
|
||||
margin: 3em 0;
|
||||
}
|
||||
|
||||
form {
|
||||
margin: 1rem 0;
|
||||
padding: 1rem;
|
||||
@ -139,16 +182,100 @@ form.samplecode-input input:focus-visible {
|
||||
background: lightgreen;
|
||||
}
|
||||
|
||||
table {
|
||||
border-top: 1px solid var(--table-border);
|
||||
border-left: 1px solid var(--table-border);
|
||||
border-spacing: 0;
|
||||
border-radius: 3px;
|
||||
.login-form {
|
||||
width: fit-content;
|
||||
margin: 3em auto;
|
||||
padding: 2em 5em;
|
||||
|
||||
border: 1px solid var(--table-border);
|
||||
border-radius: .5em;
|
||||
background: white;
|
||||
}
|
||||
|
||||
.login-form form {
|
||||
width: 20em;
|
||||
margin: 0 auto;
|
||||
display: grid;
|
||||
grid-gap: .5em;
|
||||
|
||||
border: none;
|
||||
background: none;
|
||||
}
|
||||
|
||||
.login-form form *,
|
||||
.token-form form * {
|
||||
padding: 0.5em;
|
||||
border: 1px solid var(--table-border);
|
||||
border-radius: 3px;
|
||||
}
|
||||
|
||||
.login-form form hr,
|
||||
.token-form form hr {
|
||||
padding: 0;
|
||||
width: 100%;
|
||||
}
|
||||
|
||||
.login-form button,
|
||||
.login-form a.btn,
|
||||
.token-form button {
|
||||
margin: 1em 0;
|
||||
background: var(--bg-blue);
|
||||
color: white;
|
||||
border: none;
|
||||
}
|
||||
|
||||
.border {
|
||||
padding: 1.5em;
|
||||
border: 1px solid var(--table-border);
|
||||
border-radius: .5em;
|
||||
background: white;
|
||||
}
|
||||
|
||||
table, .chart {
|
||||
border: 1px solid var(--table-border);
|
||||
padding: 1.5em;
|
||||
|
||||
border-spacing: 0;
|
||||
border-radius: .5em;
|
||||
|
||||
background: white;
|
||||
}
|
||||
|
||||
table {
|
||||
min-width: 100%;
|
||||
font-family: sans-serif;
|
||||
}
|
||||
|
||||
.border > table {
|
||||
padding: 0;
|
||||
border: none;
|
||||
background: transparent;
|
||||
}
|
||||
|
||||
.page-control {
|
||||
border-radius: .5em;
|
||||
padding: 1em 2em;
|
||||
text-align: center;
|
||||
|
||||
line-height: 1.75em;
|
||||
}
|
||||
|
||||
.page-control a {
|
||||
padding: 0 .25em;
|
||||
font-size: 1.75em;
|
||||
color: var(--bg-gray);
|
||||
text-decoration: none;
|
||||
}
|
||||
|
||||
.page-control a[href] {
|
||||
color: var(--bg-blue);
|
||||
}
|
||||
|
||||
.page-control span {
|
||||
padding: 0 .5em;
|
||||
vertical-align: text-bottom;
|
||||
}
|
||||
|
||||
#samples-table.max {
|
||||
width: 100vw;
|
||||
position: fixed;
|
||||
@ -165,64 +292,78 @@ table.samples {
|
||||
display: block;
|
||||
}
|
||||
|
||||
th {
|
||||
background: #eee;
|
||||
}
|
||||
|
||||
td, th {
|
||||
th, td {
|
||||
padding: 0.4rem .2rem;
|
||||
|
||||
border-right: 1px solid var(--table-border);
|
||||
border-bottom: 1px solid var(--table-border);
|
||||
line-height: 2em;
|
||||
|
||||
text-align: left;
|
||||
white-space: nowrap;
|
||||
vertical-align: top;
|
||||
}
|
||||
|
||||
th {
|
||||
border-bottom: 1px solid var(--bg-gray);
|
||||
}
|
||||
|
||||
td {
|
||||
font-family: monospace;
|
||||
border-bottom: 1px solid var(--bg-gray-op);
|
||||
}
|
||||
|
||||
td.bg-blue, th.bg-blue {
|
||||
tr:last-of-type > td {
|
||||
border-bottom: none;
|
||||
}
|
||||
|
||||
td > small {
|
||||
display: block;
|
||||
text-align: center;
|
||||
}
|
||||
|
||||
td.patient-id {
|
||||
width: 32em;
|
||||
text-overflow: ellipsis;
|
||||
overflow: hidden;
|
||||
display: block;
|
||||
}
|
||||
|
||||
td.bg-blue, th.bg-blue,
|
||||
td.bg-green, th.bg-green,
|
||||
td.bg-yellow, th.bg-yellow,
|
||||
td.bg-red, th.bg-red,
|
||||
td.bg-gray, th.bg-gray
|
||||
{
|
||||
width: 8em;
|
||||
}
|
||||
|
||||
td.bg-blue > small, th.bg-blue > small {
|
||||
background: var(--bg-blue);
|
||||
color: white;
|
||||
border-radius: 0.4em;
|
||||
}
|
||||
|
||||
tr:has(td.bg-blue) {
|
||||
background: var(--bg-blue-op);
|
||||
}
|
||||
|
||||
td.bg-green, th.bg-green {
|
||||
td.bg-green > small, th.bg-green > small {
|
||||
background: var(--bg-green);
|
||||
color: white;
|
||||
border-radius: 0.4em;
|
||||
}
|
||||
|
||||
tr:has(td.bg-green) {
|
||||
background: var(--bg-green-op);
|
||||
}
|
||||
|
||||
td.bg-yellow, th.bg-yellow {
|
||||
td.bg-yellow > small, th.bg-yellow > small {
|
||||
background: var(--bg-yellow);
|
||||
color: white;
|
||||
border-radius: 0.4em;
|
||||
}
|
||||
|
||||
tr:has(td.bg-yellow) {
|
||||
background: var(--bg-yellow-op);
|
||||
}
|
||||
|
||||
td.bg-red, th.bg-red {
|
||||
td.bg-red > small, th.bg-red > small {
|
||||
background: var(--bg-red);
|
||||
color: white;
|
||||
border-radius: 0.4em;
|
||||
}
|
||||
|
||||
tr:has(td.bg-red) {
|
||||
background: var(--bg-red-op);
|
||||
}
|
||||
|
||||
td.bg-gray, th.bg-gray {
|
||||
td.bg-gray > small, th.bg-gray > small {
|
||||
background: var(--bg-gray);
|
||||
color: white;
|
||||
border-radius: 0.4em;
|
||||
}
|
||||
|
||||
.bg-path {
|
||||
@ -341,12 +482,6 @@ input.inline:focus-visible {
|
||||
}
|
||||
|
||||
.chart {
|
||||
padding: 1rem;
|
||||
margin: .2rem;
|
||||
|
||||
border: 1px solid var(--table-border);
|
||||
border-radius: 3px;
|
||||
|
||||
width: calc(100% - 2.4rem - 4px);
|
||||
height: 320px;
|
||||
|
||||
@ -355,4 +490,76 @@ input.inline:focus-visible {
|
||||
|
||||
.chart-50pc {
|
||||
width: calc(50% - 2.4rem - 4px);
|
||||
}
|
||||
|
||||
.connection-display {
|
||||
display: grid;
|
||||
grid-template-columns: 10em 16em 10em;
|
||||
place-items: center;
|
||||
width: fit-content;
|
||||
margin: 1em 0;
|
||||
}
|
||||
|
||||
.connection-display > * {
|
||||
text-align: center;
|
||||
margin: auto 0;
|
||||
}
|
||||
|
||||
.connection-display .connection {
|
||||
display: block;
|
||||
width: 100%;
|
||||
height: 4px;
|
||||
background: repeating-linear-gradient(to left, white, white 2px, transparent 2px, transparent 8px, white 8px) var(--bg-red);
|
||||
}
|
||||
|
||||
.connection-display .connection.available {
|
||||
background: var(--bg-green);
|
||||
}
|
||||
|
||||
.notification {
|
||||
margin: 1em;
|
||||
padding: .5em;
|
||||
border-radius: 3px;
|
||||
text-align: center;
|
||||
}
|
||||
|
||||
.notification.success {
|
||||
color: var(--bg-green);
|
||||
}
|
||||
|
||||
.notification.error {
|
||||
color: var(--bg-red);
|
||||
}
|
||||
|
||||
a.reload {
|
||||
display: none;
|
||||
position: absolute;
|
||||
height: 1.2em;
|
||||
width: 1.2em;
|
||||
background: var(--bg-red);
|
||||
border-radius: 50%;
|
||||
|
||||
color: white;
|
||||
text-decoration: none;
|
||||
font-size: .6em;
|
||||
align-content: center;
|
||||
justify-content: center;
|
||||
}
|
||||
|
||||
.new-token {
|
||||
padding: 1em;
|
||||
background: var(--bg-green-op);
|
||||
}
|
||||
|
||||
.new-token > pre {
|
||||
margin: 0;
|
||||
border: 1px solid var(--bg-green);
|
||||
padding: .5em;
|
||||
width: max-content;
|
||||
display: inline-block;
|
||||
}
|
||||
|
||||
.no-token {
|
||||
padding: 1em;
|
||||
background: var(--bg-red-op);
|
||||
}
|
@ -10,67 +10,87 @@
|
||||
<main>
|
||||
<h1>Konfiguration</h1>
|
||||
|
||||
<h2>Allgemeine Konfiguration</h2>
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Name</th>
|
||||
<th>Wert</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<section>
|
||||
<h2>🔧 Allgemeine Konfiguration</h2>
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<td>Pseudonym erzeugt über</td>
|
||||
<td>[[ ${pseudonymGenerator} ]]</td>
|
||||
<th>Name</th>
|
||||
<th>Wert</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>Pseudonym erzeugt über</td>
|
||||
<td>[[ ${pseudonymGenerator} ]]</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>MTBFile-Sender</td>
|
||||
<td>[[ ${mtbFileSender} ]]</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td th:if="${mtbFileSender.startsWith('Rest')}">REST-Endpunkt</td>
|
||||
<td th:if="${mtbFileSender.startsWith('Kafka')}">Kafka-Broker und Topics</td>
|
||||
<td>[[ ${mtbFileEndpoint} ]]</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
</section>
|
||||
|
||||
<section th:insert="~{configs/tokens.html}">
|
||||
</section>
|
||||
|
||||
<section hx-ext="sse" th:sse-connect="@{/configs/events}">
|
||||
<div th:insert="~{configs/connectionAvailable.html}" th:hx-get="@{/configs?connectionAvailable}" hx-trigger="sse:connection-available">
|
||||
</div>
|
||||
</section>
|
||||
|
||||
<section>
|
||||
<h2><span th:if="${not transformations.isEmpty()}">✅</span><span th:if="${transformations.isEmpty()}">⛔</span> Transformationen</h2>
|
||||
|
||||
<h3>Syntax</h3>
|
||||
Hier einige Beispiele zum Syntax des JSON-Path
|
||||
<ul>
|
||||
<li style="padding: 0.6rem 0;"><span class="bg-path">diagnoses[*].icdO3T.version</span>: Ersetze die ICD-O3T-Version in allen Diagnosen, z.B. zur Version der deutschen Übersetzung</li>
|
||||
<li style="padding: 0.6rem 0;"><span class="bg-path">patient.gender</span>: Ersetze das Geschlecht des Patienten, z.B. in das von bwHC verlangte Format</li>
|
||||
</ul>
|
||||
|
||||
<h3>Konfigurierte Transformationen</h3>
|
||||
<th:block th:if="${transformations.isEmpty()}">
|
||||
<p>
|
||||
Keine konfigurierten Transformationen.
|
||||
</p>
|
||||
</th:block>
|
||||
<th:block th:if="${not transformations.isEmpty()}">
|
||||
<p>
|
||||
Hier sehen Sie eine Übersicht der konfigurierten Transformationen.
|
||||
</p>
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<td>MTBFile-Sender</td>
|
||||
<td>[[ ${mtbFileSender} ]]</td>
|
||||
<th>JSON-Path</th>
|
||||
<th>Transformation von ⇒ nach</th>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
<h2><span th:if="${connectionAvailable}">✅</span><span th:if="${not(connectionAvailable)}">⚡</span> Verbindung zum bwHC-Backend</h2>
|
||||
<p>
|
||||
Verbindung über <code>[[ ${mtbFileSender} ]]</code>. Die Verbindung ist aktuell
|
||||
<strong th:if="${connectionAvailable}" style="color: green">verfügbar.</strong>
|
||||
<strong th:if="${not(connectionAvailable)}" style="color: red">nicht verfügbar!</strong>
|
||||
</p>
|
||||
|
||||
<h2>Transformationen</h2>
|
||||
|
||||
<h3>Syntax</h3>
|
||||
Hier einige Beispiele zum Syntax des JSON-Path
|
||||
<ul>
|
||||
<li style="padding: 0.6rem 0;"><span class="bg-path">diagnoses[*].icdO3T.version</span>: Ersetze die ICD-O3T-Version in allen Diagnosen, z.B. zur Version der deutschen Übersetzung</li>
|
||||
<li style="padding: 0.6rem 0;"><span class="bg-path">patient.gender</span>: Ersetze das Geschlecht des Patienten, z.B. in das von bwHC verlangte Format</li>
|
||||
</ul>
|
||||
|
||||
<h3>Konfigurierte Transformationen</h3>
|
||||
<p>
|
||||
Hier sehen Sie eine Übersicht der konfigurierten Transformationen.
|
||||
</p>
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>JSON-Path</th>
|
||||
<th>Transformation von ⇒ nach</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr th:each="transformation : ${transformations}">
|
||||
<td>
|
||||
<span class="bg-path" title="Ersetze Wert(e) an dieser Stelle im MTB-File">[[ ${transformation.path} ]]</span>
|
||||
</td>
|
||||
<td>
|
||||
<span class="bg-from" title="Ersetze immer dann, wenn dieser Wert enthalten ist">[[ ${transformation.existingValue} ]]</span>
|
||||
<strong>⇒</strong>
|
||||
<span class="bg-to" title="Ersetze durch diesen Wert">[[ ${transformation.newValue} ]]</span>
|
||||
</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr th:each="transformation : ${transformations}">
|
||||
<td>
|
||||
<span class="bg-path" title="Ersetze Wert(e) an dieser Stelle im MTB-File">[[ ${transformation.path} ]]</span>
|
||||
</td>
|
||||
<td>
|
||||
<span class="bg-from" title="Ersetze immer dann, wenn dieser Wert enthalten ist">[[ ${transformation.existingValue} ]]</span>
|
||||
<strong>⇒</strong>
|
||||
<span class="bg-to" title="Ersetze durch diesen Wert">[[ ${transformation.newValue} ]]</span>
|
||||
</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
</th:block>
|
||||
</section>
|
||||
</main>
|
||||
<script th:src="@{/scripts.js}"></script>
|
||||
<script th:src="@{/webjars/htmx.org/dist/htmx.min.js}"></script>
|
||||
<script th:src="@{/webjars/htmx.org/dist/ext/sse.js}"></script>
|
||||
</body>
|
||||
</html>
|
@ -0,0 +1,16 @@
|
||||
<h2><span th:if="${connectionAvailable}">✅</span><span th:if="${not(connectionAvailable)}">⚡</span> Verbindung zum bwHC-Backend</h2>
|
||||
<div>
|
||||
Verbindung über <code>[[ ${mtbFileSender} ]]</code>. Die Verbindung ist aktuell
|
||||
<strong th:if="${connectionAvailable}" style="color: green">verfügbar.</strong>
|
||||
<strong th:if="${not(connectionAvailable)}" style="color: red">nicht verfügbar.</strong>
|
||||
</div>
|
||||
<div class="connection-display border">
|
||||
<img th:src="@{/server.png}" alt="ETL-Processor" />
|
||||
<span class="connection" th:classappend="${connectionAvailable ? 'available' : ''}"></span>
|
||||
<img th:if="${mtbFileSender.startsWith('Rest')}" th:src="@{/server.png}" alt="bwHC-Backend" />
|
||||
<img th:if="${mtbFileSender.startsWith('Kafka')}" th:src="@{/kafka.png}" alt="Kafka-Broker" />
|
||||
<span>ETL-Processor</span>
|
||||
<span></span>
|
||||
<span th:if="${mtbFileSender.startsWith('Rest')}">bwHC-Backend</span>
|
||||
<span th:if="${mtbFileSender.startsWith('Kafka')}">Kafka-Broker</span>
|
||||
</div>
|
39
src/main/resources/templates/configs/tokens.html
Normal file
39
src/main/resources/templates/configs/tokens.html
Normal file
@ -0,0 +1,39 @@
|
||||
<div th:if="${not tokensEnabled}">
|
||||
<h2><span>⛔</span> Tokens</h2>
|
||||
<p>Die Verwendung von Tokens ist nicht aktiviert.</p>
|
||||
</div>
|
||||
|
||||
<div id="tokens" th:if="${tokensEnabled}">
|
||||
<h2><span>✅</span> Tokens</h2>
|
||||
<div class="border">
|
||||
<div th:if="${tokens.isEmpty()}">Noch keine Tokens vorhanden.</div>
|
||||
<table th:if="${not tokens.isEmpty()}">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Name</th>
|
||||
<th>Erstellt</th>
|
||||
<th></th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr th:each="token : ${tokens}">
|
||||
<td>[[ ${token.name} ]]</td>
|
||||
<td><time th:datetime="${token.createdAt}">[[ ${token.createdAt} ]]</time></td>
|
||||
<td><button class="btn btn-red" th:hx-delete="@{/configs/tokens/{id}(id=${token.id})}" hx-target="#tokens">Löschen</button></td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
<div th:if="${newTokenValue != null and success}" class="new-token">
|
||||
Verwendung über HTTP-Basic. Bitte notieren, wird nicht erneut angezeigt: <pre>[[ ${newTokenValue} ]]</pre>
|
||||
</div>
|
||||
<div th:if="${success != null and not success}" class="no-token">
|
||||
Das Token konnte nicht erzeugt werden. Versuchen Sie einen anderen Namen.
|
||||
</div>
|
||||
<div class="token-form">
|
||||
<form th:hx-post="@{/configs/tokens}" hx-target="#tokens">
|
||||
<input placeholder="Token-Name" name="name" required />
|
||||
<button>Token Erstellen</button>
|
||||
</form>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
@ -1,5 +1,5 @@
|
||||
<!DOCTYPE html>
|
||||
<html xmlns:th="http://www.thymeleaf.org">
|
||||
<html xmlns:th="http://www.thymeleaf.org" xmlns:sec="http://www.thymeleaf.org">
|
||||
<head>
|
||||
<meta charset="UTF-8">
|
||||
<link rel="stylesheet" th:href="@{/style.css}" />
|
||||
@ -7,10 +7,27 @@
|
||||
<body>
|
||||
<div th:fragment="nav">
|
||||
<nav>
|
||||
<a class="nav-home" th:href="@{/}">
|
||||
<img th:src="@{/icon.svg}" alt="Icon" />
|
||||
<span>ETL-Processor</span>
|
||||
</a>
|
||||
<ul>
|
||||
<li><a th:href="@{/}">Übersicht</a></li>
|
||||
<li><a th:href="@{/statistics}">Statistiken</a></li>
|
||||
<li><a th:href="@{/configs}">Konfiguration</a></li>
|
||||
<li sec:authorize="hasRole('ADMIN')">
|
||||
<a th:href="@{/configs}">Konfiguration</a>
|
||||
</li>
|
||||
<li class="login" sec:authorize="not isAuthenticated()">
|
||||
<a th:href="@{/login}">Login</a>
|
||||
</li>
|
||||
<li class="login" sec:authorize="isAuthenticated()">
|
||||
<span>
|
||||
<span>👤</span>
|
||||
<span sec:authentication="name">?</span>
|
||||
</span>
|
||||
|
||||
<a th:href="@{/logout}">Abmelden</a>
|
||||
</li>
|
||||
</ul>
|
||||
</nav>
|
||||
</div>
|
||||
|
@ -1,5 +1,5 @@
|
||||
<!DOCTYPE html>
|
||||
<html lang="de" xmlns:th="http://www.thymeleaf.org">
|
||||
<html lang="de" xmlns:th="http://www.thymeleaf.org" xmlns:sec="http://www.thymeleaf.org">
|
||||
<head>
|
||||
<meta charset="UTF-8">
|
||||
<title>ETL-Prozessor</title>
|
||||
@ -9,37 +9,91 @@
|
||||
<div th:replace="~{fragments.html :: nav}"></div>
|
||||
<main>
|
||||
|
||||
<h1>Letzte Anfragen</h1>
|
||||
<h1>Alle Anfragen<a id="reload-notify" class="reload" title="Neue Anfragen" th:href="@{/}">⟳</a></h1>
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Status</th>
|
||||
<th>Typ</th>
|
||||
<th>ID</th>
|
||||
<th>Datum</th>
|
||||
<th>Patienten-ID</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr th:each="request : ${requests}">
|
||||
<td th:if="${request.status.value.contains('success')}" class="bg-green"><small>[[ ${request.status} ]]</small></td>
|
||||
<td th:if="${request.status.value.contains('warning')}" class="bg-yellow"><small>[[ ${request.status} ]]</small></td>
|
||||
<td th:if="${request.status.value.contains('error')}" class="bg-red"><small>[[ ${request.status} ]]</small></td>
|
||||
<td th:if="${request.status.value == 'unknown'}" class="bg-gray"><small>[[ ${request.status} ]]</small></td>
|
||||
<td th:if="${request.status.value == 'duplication'}" class="bg-gray"><small>[[ ${request.status} ]]</small></td>
|
||||
<td th:style="${request.type.value == 'delete'} ? 'color: red;'"><small>[[ ${request.type} ]]</small></td>
|
||||
<td th:if="not ${request.report}">[[ ${request.uuid} ]]</td>
|
||||
<td th:if="${request.report}">
|
||||
<a th:href="@{/report/{id}(id=${request.uuid})}">[[ ${request.uuid} ]]</a>
|
||||
</td>
|
||||
<td><time th:datetime="${request.processedAt}">[[ ${request.processedAt} ]]</time></td>
|
||||
<td>[[ ${request.patientId} ]]</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
<div>
|
||||
<h2 th:if="${patientId != null}">
|
||||
Betreffend Patienten-Pseudonym <span class="monospace" th:text="${patientId}">***</span>
|
||||
<a class="btn btn-blue" th:if="${patientId != null}" th:href="@{/}">Alle anzeigen</a>
|
||||
</h2>
|
||||
</div>
|
||||
|
||||
<div class="border">
|
||||
<div th:if="${patientId == null}" class="page-control">
|
||||
<a id="first-page-link" th:href="@{/(page=${0})}" title="Zum Anfang: Taste W" th:if="${not requests.isFirst()}">⇤</a><a th:if="${requests.isFirst()}">⇤</a>
|
||||
<a id="prev-page-link" th:href="@{/(page=${requests.getNumber() - 1})}" title="Seite zurück: Taste A" th:if="${not requests.isFirst()}">←</a><a th:if="${requests.isFirst()}">←</a>
|
||||
<span>Seite [[ ${requests.getNumber() + 1} ]] von [[ ${requests.getTotalPages()} ]]</span>
|
||||
<a id="next-page-link" th:href="@{/(page=${requests.getNumber() + 1})}" title="Seite vor: Taste D" th:if="${not requests.isLast()}">→</a><a th:if="${requests.isLast()}">→</a>
|
||||
<a id="last-page-link" th:href="@{/(page=${requests.getTotalPages() - 1})}" title="Zum Ende: Taste S" th:if="${not requests.isLast()}">⇥</a><a th:if="${requests.isLast()}">⇥</a>
|
||||
</div>
|
||||
<div th:if="${patientId != null}" class="page-control">
|
||||
<a id="first-page-link" th:href="@{/patient/{patientId}(patientId=${patientId},page=${0})}" title="Zum Anfang: Taste W" th:if="${not requests.isFirst()}">⇤</a><a th:if="${requests.isFirst()}">⇤</a>
|
||||
<a id="prev-page-link" th:href="@{/patient/{patientId}(patientId=${patientId},page=${requests.getNumber() - 1})}" title="Seite zurück: Taste A" th:if="${not requests.isFirst()}">←</a><a th:if="${requests.isFirst()}">←</a>
|
||||
<span>Seite [[ ${requests.getNumber() + 1} ]] von [[ ${requests.getTotalPages()} ]]</span>
|
||||
<a id="next-page-link" th:href="@{/patient/{patientId}(patientId=${patientId},page=${requests.getNumber() + 1})}" title="Seite vor: Taste D" th:if="${not requests.isLast()}">→</a><a th:if="${requests.isLast()}">→</a>
|
||||
<a id="last-page-link" th:href="@{/patient/{patientId}(patientId=${patientId},page=${requests.getTotalPages() - 1})}" title="Zum Ende: Taste S" th:if="${not requests.isLast()}">⇥</a><a th:if="${requests.isLast()}">⇥</a>
|
||||
</div>
|
||||
<table class="paged">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Status</th>
|
||||
<th>Typ</th>
|
||||
<th>ID</th>
|
||||
<th>Datum</th>
|
||||
<th>Patienten-ID</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr th:each="request : ${requests}">
|
||||
<td th:if="${request.status.value.contains('success')}" class="bg-green"><small>[[ ${request.status} ]]</small></td>
|
||||
<td th:if="${request.status.value.contains('warning')}" class="bg-yellow"><small>[[ ${request.status} ]]</small></td>
|
||||
<td th:if="${request.status.value.contains('error')}" class="bg-red"><small>[[ ${request.status} ]]</small></td>
|
||||
<td th:if="${request.status.value == 'unknown'}" class="bg-gray"><small>[[ ${request.status} ]]</small></td>
|
||||
<td th:if="${request.status.value == 'duplication'}" class="bg-gray"><small>[[ ${request.status} ]]</small></td>
|
||||
<td th:style="${request.type.value == 'delete'} ? 'color: red;'"><small>[[ ${request.type} ]]</small></td>
|
||||
<td th:if="not ${request.report}">[[ ${request.uuid} ]]</td>
|
||||
<td th:if="${request.report}">
|
||||
<th:block sec:authorize="not authenticated">[[ ${request.uuid} ]]</th:block>
|
||||
<a th:href="@{/report/{id}(id=${request.uuid})}" sec:authorize="authenticated">[[ ${request.uuid} ]]</a>
|
||||
</td>
|
||||
<td><time th:datetime="${request.processedAt}">[[ ${request.processedAt} ]]</time></td>
|
||||
<td class="patient-id" th:if="${patientId != null}" sec:authorize="authenticated">
|
||||
[[ ${request.patientId} ]]
|
||||
</td>
|
||||
<td class="patient-id" th:if="${patientId == null}" sec:authorize="authenticated">
|
||||
<a th:href="@{/patient/{pid}(pid=${request.patientId})}">[[ ${request.patientId} ]]</a>
|
||||
</td>
|
||||
<td class="patient-id" sec:authorize="not authenticated">***</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
</div>
|
||||
|
||||
</main>
|
||||
<script th:src="@{/scripts.js}"></script>
|
||||
<script>
|
||||
window.addEventListener('load', () => {
|
||||
let keyBindings = {
|
||||
'w': 'first-page-link',
|
||||
'a': 'prev-page-link',
|
||||
'd': 'next-page-link',
|
||||
's': 'last-page-link'
|
||||
};
|
||||
window.onkeydown = (event) => {
|
||||
for (const [key, elemId] of Object.entries(keyBindings)) {
|
||||
if (event.key === key && document.getElementById(elemId)) {
|
||||
document.getElementById(elemId).style.background = 'yellow';
|
||||
document.getElementById(elemId).click();
|
||||
}
|
||||
}
|
||||
};
|
||||
});
|
||||
|
||||
const eventSource = new EventSource('statistics/events');
|
||||
eventSource.addEventListener('newrequest', event => {
|
||||
console.log(event);
|
||||
document.getElementById('reload-notify').style.display = 'inline-flex';
|
||||
});
|
||||
</script>
|
||||
</body>
|
||||
</html>
|
25
src/main/resources/templates/login.html
Normal file
25
src/main/resources/templates/login.html
Normal file
@ -0,0 +1,25 @@
|
||||
<!DOCTYPE html>
|
||||
<html lang="de" xmlns:th="http://www.thymeleaf.org">
|
||||
<head>
|
||||
<meta charset="UTF-8">
|
||||
<title>ETL-Prozessor</title>
|
||||
<link rel="stylesheet" th:href="@{/style.css}" />
|
||||
</head>
|
||||
<body>
|
||||
<div th:replace="~{fragments.html :: nav}"></div>
|
||||
<main>
|
||||
<div class="login-form">
|
||||
<h2 class="centered">Anmelden</h2>
|
||||
<div class="centered notification error" th:if="${param.error}">Anmeldung nicht erfolgreich</div>
|
||||
<div class="centered notification success" th:if="${param.logout}">Sie haben sich abgemeldet</div>
|
||||
<form method="post" th:action="@{/login}">
|
||||
<input type="text" id="username" name="username" class="form-control" placeholder="Username" required="" autofocus="" />
|
||||
<input type="password" id="password" name="password" class="form-control" placeholder="Password" required="" />
|
||||
<button type="submit">Anmelden</button>
|
||||
<hr th:if="${not oidcLogins.isEmpty()}" />
|
||||
<a th:each="oidcLogin : ${oidcLogins}" class="btn" th:href="@{/oauth2/authorization/{provider}(provider=${oidcLogin.component1()})}">OIDC Login - [[ ${oidcLogin.component2()} ]]</a>
|
||||
</form>
|
||||
</div>
|
||||
</main>
|
||||
</body>
|
||||
</html>
|
@ -1,5 +1,5 @@
|
||||
<!DOCTYPE html>
|
||||
<html lang="de" xmlns:th="http://www.thymeleaf.org">
|
||||
<html lang="de" xmlns:th="http://www.thymeleaf.org" xmlns:sec="http://www.thymeleaf.org">
|
||||
<head>
|
||||
<meta charset="UTF-8">
|
||||
<title>ETL-Prozessor</title>
|
||||
@ -15,6 +15,7 @@
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Status</th>
|
||||
<th>Typ</th>
|
||||
<th>ID</th>
|
||||
<th>Datum</th>
|
||||
<th>Patienten-ID</th>
|
||||
@ -27,21 +28,27 @@
|
||||
<td th:if="${request.status.value == 'error'}" class="bg-red"><small>[[ ${request.status} ]]</small></td>
|
||||
<td th:if="${request.status.value == 'unknown'}" class="bg-gray"><small>[[ ${request.status} ]]</small></td>
|
||||
<td th:if="${request.status.value == 'duplication'}" class="bg-gray"><small>[[ ${request.status} ]]</small></td>
|
||||
<td th:style="${request.type.value == 'delete'} ? 'color: red;'"><small>[[ ${request.type} ]]</small></td>
|
||||
<td>[[ ${request.uuid} ]]</td>
|
||||
<td><time th:datetime="${request.processedAt}">[[ ${request.processedAt} ]]</time></td>
|
||||
<td>[[ ${request.patientId} ]]</td>
|
||||
<td class="patient-id" sec:authorize="authenticated">[[ ${request.patientId} ]]</td>
|
||||
<td class="patient-id" sec:authorize="not authenticated">***</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
<h2 th:text="${request.report.description}"></h2>
|
||||
|
||||
<table th:if="not ${issues.isEmpty()}">
|
||||
<p th:if="${issues.isEmpty()}">
|
||||
Keine weiteren Angaben.
|
||||
</p>
|
||||
|
||||
<table th:if="${not issues.isEmpty()}">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Schweregrad</th>
|
||||
<th>Beschreibung</th>
|
||||
</tr>
|
||||
<tr>
|
||||
<th>Schweregrad</th>
|
||||
<th>Beschreibung</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr th:each="issue : ${issues}">
|
||||
|
@ -13,28 +13,32 @@
|
||||
Hier sehen Sie eine Übersicht über eingegangene Anfragen.
|
||||
</p>
|
||||
|
||||
<h2>MTB-File-Anfragen</h2>
|
||||
<p>
|
||||
Anfragen zur Aktualisierung von Patientendaten durch Übermittlung eines MTB-Files.
|
||||
</p>
|
||||
<div>
|
||||
<div id="piechart1" class="chart chart-50pc"></div>
|
||||
<div id="piechart2" class="chart chart-50pc"></div>
|
||||
</div>
|
||||
<div id="barchart" class="chart"></div>
|
||||
<section>
|
||||
<h2>MTB-File-Anfragen</h2>
|
||||
<p>
|
||||
Anfragen zur Aktualisierung von Patientendaten durch Übermittlung eines MTB-Files.
|
||||
</p>
|
||||
<div>
|
||||
<div id="piechart1" class="chart chart-50pc"></div>
|
||||
<div id="piechart2" class="chart chart-50pc"></div>
|
||||
</div>
|
||||
<div id="barchart" class="chart"></div>
|
||||
</section>
|
||||
|
||||
<h2>Löschanfragen</h2>
|
||||
<p>
|
||||
Anfragen zur Löschung von Patientendaten, wenn kein Consent vorliegt.
|
||||
</p>
|
||||
<div>
|
||||
<div id="piechartdel1" class="chart chart-50pc"></div>
|
||||
<div id="piechartdel2" class="chart chart-50pc"></div>
|
||||
</div>
|
||||
<div id="barchartdel" class="chart"></div>
|
||||
<section>
|
||||
<h2>Löschanfragen</h2>
|
||||
<p>
|
||||
Anfragen zur Löschung von Patientendaten, wenn kein Consent vorliegt.
|
||||
</p>
|
||||
<div>
|
||||
<div id="piechartdel1" class="chart chart-50pc"></div>
|
||||
<div id="piechartdel2" class="chart chart-50pc"></div>
|
||||
</div>
|
||||
<div id="barchartdel" class="chart"></div>
|
||||
</section>
|
||||
|
||||
</main>
|
||||
<script th:src="@{/echarts.min.js}"></script>
|
||||
<script th:src="@{/webjars/echarts/dist/echarts.min.js}"></script>
|
||||
<script th:src="@{/scripts.js}"></script>
|
||||
<script>
|
||||
window.onload = () => {
|
||||
|
@ -1,7 +1,7 @@
|
||||
/*
|
||||
* This file is part of ETL-Processor
|
||||
*
|
||||
* Copyright (c) 2023 Comprehensive Cancer Center Mainfranken, Datenintegrationszentrum Philipps-Universität Marburg and Contributors
|
||||
* Copyright (c) 2024 Comprehensive Cancer Center Mainfranken, Datenintegrationszentrum Philipps-Universität Marburg and Contributors
|
||||
*
|
||||
* This program is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
@ -35,6 +35,8 @@ import org.mockito.junit.jupiter.MockitoExtension
|
||||
import org.mockito.kotlin.*
|
||||
import org.springframework.kafka.core.KafkaTemplate
|
||||
import org.springframework.kafka.support.SendResult
|
||||
import org.springframework.retry.policy.SimpleRetryPolicy
|
||||
import org.springframework.retry.support.RetryTemplateBuilder
|
||||
import java.util.concurrent.CompletableFuture.completedFuture
|
||||
import java.util.concurrent.ExecutionException
|
||||
|
||||
@ -52,10 +54,12 @@ class KafkaMtbFileSenderTest {
|
||||
@Mock kafkaTemplate: KafkaTemplate<String, String>
|
||||
) {
|
||||
val kafkaTargetProperties = KafkaTargetProperties("testtopic")
|
||||
val retryTemplate = RetryTemplateBuilder().customPolicy(SimpleRetryPolicy(1)).build()
|
||||
|
||||
this.objectMapper = ObjectMapper()
|
||||
this.kafkaTemplate = kafkaTemplate
|
||||
|
||||
this.kafkaMtbFileSender = KafkaMtbFileSender(kafkaTemplate, kafkaTargetProperties, objectMapper)
|
||||
this.kafkaMtbFileSender = KafkaMtbFileSender(kafkaTemplate, kafkaTargetProperties, retryTemplate, objectMapper)
|
||||
}
|
||||
|
||||
@ParameterizedTest
|
||||
@ -118,6 +122,58 @@ class KafkaMtbFileSenderTest {
|
||||
assertThat(captor.secondValue).isEqualTo(objectMapper.writeValueAsString(kafkaRecordData("TestID", Consent.Status.REJECTED)))
|
||||
}
|
||||
|
||||
@ParameterizedTest
|
||||
@MethodSource("requestWithResponseSource")
|
||||
fun shouldRetryOnMtbFileKafkaSendError(testData: TestData) {
|
||||
val kafkaTargetProperties = KafkaTargetProperties("testtopic")
|
||||
val retryTemplate = RetryTemplateBuilder().customPolicy(SimpleRetryPolicy(3)).build()
|
||||
this.kafkaMtbFileSender = KafkaMtbFileSender(this.kafkaTemplate, kafkaTargetProperties, retryTemplate, this.objectMapper)
|
||||
|
||||
doAnswer {
|
||||
if (null != testData.exception) {
|
||||
throw testData.exception
|
||||
}
|
||||
completedFuture(SendResult<String, String>(null, null))
|
||||
}.whenever(kafkaTemplate).send(anyString(), anyString(), anyString())
|
||||
|
||||
kafkaMtbFileSender.send(MtbFileSender.MtbFileRequest("TestID", mtbFile(Consent.Status.ACTIVE)))
|
||||
|
||||
val expectedCount = when (testData.exception) {
|
||||
// OK - No Retry
|
||||
null -> times(1)
|
||||
// Request failed - Retry max 3 times
|
||||
else -> times(3)
|
||||
}
|
||||
|
||||
verify(kafkaTemplate, expectedCount).send(anyString(), anyString(), anyString())
|
||||
}
|
||||
|
||||
@ParameterizedTest
|
||||
@MethodSource("requestWithResponseSource")
|
||||
fun shouldRetryOnDeleteKafkaSendError(testData: TestData) {
|
||||
val kafkaTargetProperties = KafkaTargetProperties("testtopic")
|
||||
val retryTemplate = RetryTemplateBuilder().customPolicy(SimpleRetryPolicy(3)).build()
|
||||
this.kafkaMtbFileSender = KafkaMtbFileSender(this.kafkaTemplate, kafkaTargetProperties, retryTemplate, this.objectMapper)
|
||||
|
||||
doAnswer {
|
||||
if (null != testData.exception) {
|
||||
throw testData.exception
|
||||
}
|
||||
completedFuture(SendResult<String, String>(null, null))
|
||||
}.whenever(kafkaTemplate).send(anyString(), anyString(), anyString())
|
||||
|
||||
kafkaMtbFileSender.send(MtbFileSender.DeleteRequest("TestID", "PID"))
|
||||
|
||||
val expectedCount = when (testData.exception) {
|
||||
// OK - No Retry
|
||||
null -> times(1)
|
||||
// Request failed - Retry max 3 times
|
||||
else -> times(3)
|
||||
}
|
||||
|
||||
verify(kafkaTemplate, expectedCount).send(anyString(), anyString(), anyString())
|
||||
}
|
||||
|
||||
companion object {
|
||||
fun mtbFile(consentStatus: Consent.Status): MtbFile {
|
||||
return if (consentStatus == Consent.Status.ACTIVE) {
|
||||
|
@ -1,7 +1,7 @@
|
||||
/*
|
||||
* This file is part of ETL-Processor
|
||||
*
|
||||
* Copyright (c) 2023 Comprehensive Cancer Center Mainfranken, Datenintegrationszentrum Philipps-Universität Marburg and Contributors
|
||||
* Copyright (c) 2024 Comprehensive Cancer Center Mainfranken, Datenintegrationszentrum Philipps-Universität Marburg and Contributors
|
||||
*
|
||||
* This program is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
@ -28,6 +28,9 @@ import org.junit.jupiter.params.ParameterizedTest
|
||||
import org.junit.jupiter.params.provider.MethodSource
|
||||
import org.springframework.http.HttpMethod
|
||||
import org.springframework.http.HttpStatus
|
||||
import org.springframework.retry.policy.SimpleRetryPolicy
|
||||
import org.springframework.retry.support.RetryTemplateBuilder
|
||||
import org.springframework.test.web.client.ExpectedCount
|
||||
import org.springframework.test.web.client.MockRestServiceServer
|
||||
import org.springframework.test.web.client.match.MockRestRequestMatchers.method
|
||||
import org.springframework.test.web.client.match.MockRestRequestMatchers.requestTo
|
||||
@ -44,10 +47,11 @@ class RestMtbFileSenderTest {
|
||||
fun setup() {
|
||||
val restTemplate = RestTemplate()
|
||||
val restTargetProperties = RestTargetProperties("http://localhost:9000/mtbfile")
|
||||
val retryTemplate = RetryTemplateBuilder().customPolicy(SimpleRetryPolicy(1)).build()
|
||||
|
||||
this.mockRestServiceServer = MockRestServiceServer.createServer(restTemplate)
|
||||
|
||||
this.restMtbFileSender = RestMtbFileSender(restTemplate, restTargetProperties)
|
||||
this.restMtbFileSender = RestMtbFileSender(restTemplate, restTargetProperties, retryTemplate)
|
||||
}
|
||||
|
||||
@ParameterizedTest
|
||||
@ -80,6 +84,64 @@ class RestMtbFileSenderTest {
|
||||
assertThat(response.body).isEqualTo(requestWithResponse.response.body)
|
||||
}
|
||||
|
||||
@ParameterizedTest
|
||||
@MethodSource("mtbFileRequestWithResponseSource")
|
||||
fun shouldRetryOnMtbFileHttpRequestError(requestWithResponse: RequestWithResponse) {
|
||||
val restTemplate = RestTemplate()
|
||||
val restTargetProperties = RestTargetProperties("http://localhost:9000/mtbfile")
|
||||
val retryTemplate = RetryTemplateBuilder().customPolicy(SimpleRetryPolicy(3)).build()
|
||||
|
||||
this.mockRestServiceServer = MockRestServiceServer.createServer(restTemplate)
|
||||
this.restMtbFileSender = RestMtbFileSender(restTemplate, restTargetProperties, retryTemplate)
|
||||
|
||||
val expectedCount = when (requestWithResponse.httpStatus) {
|
||||
// OK - No Retry
|
||||
HttpStatus.OK, HttpStatus.CREATED -> ExpectedCount.max(1)
|
||||
// Request failed - Retry max 3 times
|
||||
else -> ExpectedCount.max(3)
|
||||
}
|
||||
|
||||
this.mockRestServiceServer.expect(expectedCount) {
|
||||
method(HttpMethod.POST)
|
||||
requestTo("/mtbfile")
|
||||
}.andRespond {
|
||||
withStatus(requestWithResponse.httpStatus).body(requestWithResponse.body).createResponse(it)
|
||||
}
|
||||
|
||||
val response = restMtbFileSender.send(MtbFileSender.MtbFileRequest("TestID", mtbFile))
|
||||
assertThat(response.status).isEqualTo(requestWithResponse.response.status)
|
||||
assertThat(response.body).isEqualTo(requestWithResponse.response.body)
|
||||
}
|
||||
|
||||
@ParameterizedTest
|
||||
@MethodSource("deleteRequestWithResponseSource")
|
||||
fun shouldRetryOnDeleteHttpRequestError(requestWithResponse: RequestWithResponse) {
|
||||
val restTemplate = RestTemplate()
|
||||
val restTargetProperties = RestTargetProperties("http://localhost:9000/mtbfile")
|
||||
val retryTemplate = RetryTemplateBuilder().customPolicy(SimpleRetryPolicy(3)).build()
|
||||
|
||||
this.mockRestServiceServer = MockRestServiceServer.createServer(restTemplate)
|
||||
this.restMtbFileSender = RestMtbFileSender(restTemplate, restTargetProperties, retryTemplate)
|
||||
|
||||
val expectedCount = when (requestWithResponse.httpStatus) {
|
||||
// OK - No Retry
|
||||
HttpStatus.OK, HttpStatus.CREATED -> ExpectedCount.max(1)
|
||||
// Request failed - Retry max 3 times
|
||||
else -> ExpectedCount.max(3)
|
||||
}
|
||||
|
||||
this.mockRestServiceServer.expect(expectedCount) {
|
||||
method(HttpMethod.DELETE)
|
||||
requestTo("/mtbfile")
|
||||
}.andRespond {
|
||||
withStatus(requestWithResponse.httpStatus).body(requestWithResponse.body).createResponse(it)
|
||||
}
|
||||
|
||||
val response = restMtbFileSender.send(MtbFileSender.DeleteRequest("TestID", "PID"))
|
||||
assertThat(response.status).isEqualTo(requestWithResponse.response.status)
|
||||
assertThat(response.body).isEqualTo(requestWithResponse.response.body)
|
||||
}
|
||||
|
||||
companion object {
|
||||
data class RequestWithResponse(
|
||||
val httpStatus: HttpStatus,
|
||||
|
@ -0,0 +1,154 @@
|
||||
/*
|
||||
* This file is part of ETL-Processor
|
||||
*
|
||||
* Copyright (c) 2024 Comprehensive Cancer Center Mainfranken, Datenintegrationszentrum Philipps-Universität Marburg and Contributors
|
||||
*
|
||||
* This program is free software: you can redistribute it and/or modify
|
||||
* it under the terms of the GNU Affero General Public License as published
|
||||
* by the Free Software Foundation, either version 3 of the License, or
|
||||
* (at your option) any later version.
|
||||
*
|
||||
* This program is distributed in the hope that it will be useful,
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||
* GNU Affero General Public License for more details.
|
||||
*
|
||||
* You should have received a copy of the GNU Affero General Public License
|
||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||
*/
|
||||
|
||||
package dev.dnpm.etl.processor.services
|
||||
|
||||
import org.assertj.core.api.Assertions.assertThat
|
||||
import org.junit.jupiter.api.BeforeEach
|
||||
import org.junit.jupiter.api.Test
|
||||
import org.junit.jupiter.api.extension.ExtendWith
|
||||
import org.mockito.ArgumentCaptor
|
||||
import org.mockito.ArgumentMatchers.anyLong
|
||||
import org.mockito.ArgumentMatchers.anyString
|
||||
import org.mockito.Mock
|
||||
import org.mockito.junit.jupiter.MockitoExtension
|
||||
import org.mockito.kotlin.*
|
||||
import org.springframework.security.crypto.password.PasswordEncoder
|
||||
import org.springframework.security.provisioning.InMemoryUserDetailsManager
|
||||
import java.util.*
|
||||
import java.util.function.Consumer
|
||||
|
||||
@ExtendWith(MockitoExtension::class)
|
||||
class TokenServiceTest {
|
||||
|
||||
private lateinit var userDetailsManager: InMemoryUserDetailsManager
|
||||
private lateinit var passwordEncoder: PasswordEncoder
|
||||
private lateinit var tokenRepository: TokenRepository
|
||||
|
||||
private lateinit var tokenService: TokenService
|
||||
|
||||
@BeforeEach
|
||||
fun setup(
|
||||
@Mock userDetailsManager: InMemoryUserDetailsManager,
|
||||
@Mock passwordEncoder: PasswordEncoder,
|
||||
@Mock tokenRepository: TokenRepository
|
||||
) {
|
||||
this.userDetailsManager = userDetailsManager
|
||||
this.passwordEncoder = passwordEncoder
|
||||
this.tokenRepository = tokenRepository
|
||||
|
||||
this.tokenService = TokenService(userDetailsManager, passwordEncoder, tokenRepository)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun shouldEncodePasswordForNewToken() {
|
||||
doAnswer { "{test}verysecret" }.whenever(passwordEncoder).encode(anyString())
|
||||
|
||||
val actual = this.tokenService.addToken("Test Token")
|
||||
|
||||
assertThat(actual).satisfies(
|
||||
Consumer { assertThat(it.isSuccess).isTrue() },
|
||||
Consumer { assertThat(it.getOrNull()).matches("testtoken:[A-Za-z0-9]{48}$") }
|
||||
)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun shouldContainAlphanumTokenUserPart() {
|
||||
doAnswer { "{test}verysecret" }.whenever(passwordEncoder).encode(anyString())
|
||||
|
||||
val actual = this.tokenService.addToken("Test Token")
|
||||
|
||||
assertThat(actual).satisfies(
|
||||
Consumer { assertThat(it.isSuccess).isTrue() },
|
||||
Consumer { assertThat(it.getOrDefault("")).startsWith("testtoken:") }
|
||||
)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun shouldNotAllowSameTokenUserPartTwice() {
|
||||
doReturn(true).whenever(userDetailsManager).userExists(anyString())
|
||||
|
||||
val actual = this.tokenService.addToken("Test Token")
|
||||
|
||||
assertThat(actual).satisfies(Consumer { assertThat(it.isFailure).isTrue() })
|
||||
verify(tokenRepository, never()).save(any())
|
||||
}
|
||||
|
||||
@Test
|
||||
fun shouldSaveNewToken() {
|
||||
doAnswer { "{test}verysecret" }.whenever(passwordEncoder).encode(anyString())
|
||||
|
||||
val actual = this.tokenService.addToken("Test Token")
|
||||
|
||||
val captor = ArgumentCaptor.forClass(Token::class.java)
|
||||
verify(tokenRepository, times(1)).save(captor.capture())
|
||||
|
||||
assertThat(actual).satisfies(Consumer { assertThat(it.isSuccess).isTrue() })
|
||||
assertThat(captor.value).satisfies(
|
||||
Consumer { assertThat(it.name).isEqualTo("Test Token") },
|
||||
Consumer { assertThat(it.username).isEqualTo("testtoken") },
|
||||
Consumer { assertThat(it.password).isEqualTo("{test}verysecret") }
|
||||
)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun shouldDeleteExistingToken() {
|
||||
doAnswer {
|
||||
val id = it.arguments[0] as Long
|
||||
Optional.of(Token(id, "Test Token", "testtoken", "{test}hsdajfgadskjhfgsdkfjg"))
|
||||
}.whenever(tokenRepository).findById(anyLong())
|
||||
|
||||
this.tokenService.deleteToken(42)
|
||||
|
||||
val stringCaptor = ArgumentCaptor.forClass(String::class.java)
|
||||
verify(userDetailsManager, times(1)).deleteUser(stringCaptor.capture())
|
||||
assertThat(stringCaptor.value).isEqualTo("testtoken")
|
||||
|
||||
val tokenCaptor = ArgumentCaptor.forClass(Token::class.java)
|
||||
verify(tokenRepository, times(1)).delete(tokenCaptor.capture())
|
||||
assertThat(tokenCaptor.value.id).isEqualTo(42)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun shouldReturnAllTokensFromRepository() {
|
||||
val expected = listOf(
|
||||
Token(1, "Test Token 1", "testtoken1", "{test}hsdajfgadskjhfgsdkfjg"),
|
||||
Token(2, "Test Token 2", "testtoken2", "{test}asdasdasdasdasdasdasd")
|
||||
)
|
||||
|
||||
doReturn(expected).whenever(tokenRepository).findAll()
|
||||
|
||||
assertThat(tokenService.findAll()).isEqualTo(expected)
|
||||
}
|
||||
|
||||
@Test
|
||||
fun shouldAddAllTokensFromRepositoryToUserDataManager() {
|
||||
val expected = listOf(
|
||||
Token(1, "Test Token 1", "testtoken1", "{test}hsdajfgadskjhfgsdkfjg"),
|
||||
Token(2, "Test Token 2", "testtoken2", "{test}asdasdasdasdasdasdasd")
|
||||
)
|
||||
|
||||
doReturn(expected).whenever(tokenRepository).findAll()
|
||||
|
||||
tokenService.setup()
|
||||
|
||||
verify(userDetailsManager, times(expected.size)).createUser(any())
|
||||
}
|
||||
|
||||
}
|
Reference in New Issue
Block a user