1
0
mirror of https://github.com/pcvolkmer/etl-processor.git synced 2025-07-01 22:22:53 +00:00

34 Commits

Author SHA1 Message Date
a31d2b4bcc build: bump version 2024-02-05 07:47:17 +01:00
67d5fb4c67 docs: mention quality report page access restriction 2024-02-05 07:29:47 +01:00
329be65d1a feat: forbid access to report if not logged in 2024-02-05 07:18:31 +01:00
91fe3d1c23 docs: add example login image 2024-02-01 18:30:02 +01:00
f4b86ce2ea docs: add OIDC configuration options to README.md 2024-02-01 18:28:33 +01:00
19d0daa442 docs: move README.md to bindings folder 2024-02-01 17:00:16 +01:00
cc9811d11d docs: move README.md to bindings folder 2024-02-01 16:59:23 +01:00
8ce5b06823 fix: make security config optional for login controller 2024-02-01 16:54:41 +01:00
3cc34fb30b feat: usage of CA certificate files within image/container 2024-02-01 16:45:22 +01:00
17e04a3f89 feat: add basic support for OIDC login 2024-01-31 15:57:16 +01:00
f71a775e12 chore: update spring boot to version 3.2.2 2024-01-23 01:04:35 +01:00
45c83e943b docs: Add information about other reference IDs anonymization 2024-01-22 10:33:27 +01:00
6dcbfde62e test: add tests for TokenService 2024-01-21 14:13:09 +01:00
4cdc419478 test: add test to ensure redirect of not logged in 2024-01-20 19:35:40 +01:00
90b529adb4 refactor: move test class to related package 2024-01-20 19:16:52 +01:00
a3bc60986b test: add security related tests for MtbFileRestController 2024-01-19 14:11:03 +01:00
f5df0b5d22 test: add tests to ensure TokenService is present if required 2024-01-19 13:10:36 +01:00
972ac745e9 fix: add missing token screenshot 2024-01-18 14:44:44 +01:00
358373cf70 Merge pull request #30 from CCC-MF/issue_29
Issue #29: Unterstützung für Endpoint-Tokens
2024-01-18 14:29:52 +01:00
27a62321fa docs: add documentation about token usage 2024-01-18 14:26:09 +01:00
30cf0fd22e feat #29: add initial support for mtbfile api tokens 2024-01-18 14:13:15 +01:00
531a8589db feat: push connection available state to client 2024-01-17 14:32:42 +01:00
fa89a64ddd Merge pull request #28 from CCC-MF/issue_24
feat #24: use htmx to refresh connection status every 20s
2024-01-17 12:35:35 +01:00
45ad5e8827 feat #24: use htmx to refresh connection status every 20s 2024-01-17 12:27:44 +01:00
c4eb4d0fe2 feat #25: add link to requests related to patient pseudonyme (#27) 2024-01-15 10:26:56 +01:00
4bc69a353c Merge pull request #26 from CCC-MF/issue_23
feat #23: add reload button to display on new request
2024-01-15 10:15:36 +01:00
9d30f750f7 feat #23: add reload button to display on new request 2024-01-15 09:17:38 +01:00
a1a252d5a9 build: use webjars for JS dependencies for now 2024-01-15 07:18:14 +01:00
568942bfe5 fix: typo in README.md 2024-01-14 17:31:30 +01:00
15f0432553 test: ensure configured generator bean is created 2024-01-12 21:27:55 +01:00
113bf2dd2e test: add pseudonymize generator property and default to tests 2024-01-12 19:59:01 +01:00
7ac151202a refactor: Use config new pseudonym generator config param
This deprecates the old param:
* `APP_PSEUDONYMIZER`: deprecated
* `APP_PSEUDONYM_GENERATOR`: has precedence
2024-01-12 16:55:18 +01:00
5d9d47c2df fix: append css class, not css style 2024-01-12 13:49:54 +01:00
585468314c feat: add admin credentials to deploy folder 2024-01-11 16:34:03 +01:00
40 changed files with 1153 additions and 106 deletions

2
.gitignore vendored
View File

@ -5,6 +5,8 @@ build/
!**/src/main/**/build/
!**/src/test/**/build/
bindings/ca-certificates/*.pem
### STS ###
.apt_generated
.classpath

View File

@ -36,6 +36,15 @@ Ist diese nicht gesetzt. wird intern eine Anonymisierung der Patienten-ID vorgen
* `APP_PSEUDONYMIZE_PREFIX`: Standortbezogenes Prefix - `UNKNOWN`, wenn nicht gesetzt
* `APP_PSEUDONYMIZE_GENERATOR`: `BUILDIN` oder `GPAS` - `BUILDIN`, wenn nicht gesetzt
**Hinweise**:
* Der alte Konfigurationsparameter `APP_PSEUDONYMIZER` mit den Werten `GPAS` oder `BUILDIN` sollte nicht
mehr verwendet werden.
* Die Pseudonymisierung erfolgt im ETL-Prozessor nur für die Patienten-ID.
Andere Referenz-IDs werden nicht anonymisiert.
Dies erfolgt bei Nutzung von **[onkostar-plugin-dnpmexport](https://github.com/CCC-MF/onkostar-plugin-dnpmexport)**
bereits im Plugin selbst.
#### Eingebaute Anonymisierung
Wurde keine oder die Verwendung der eingebauten Anonymisierung konfiguriert, so wird für die Patienten-ID der
@ -71,13 +80,59 @@ Hier Beispiele für das Beispielpasswort `very-secret`:
Wird kein Administrator-Passwort angegeben, wird ein zufälliger Wert generiert und beim Start der Anwendung in den Logs
angezeigt.
#### Weitere (nicht administrative) Nutzer mit OpenID Connect
Die folgenden Konfigurationsparameter werden benötigt, um die Authentifizierung weiterer Benutzer an einen OIDC-Provider
zu delegieren.
Ein Admin-Benutzer muss dabei konfiguriert sein.
* `APP_SECURITY_ENABLE_OIDC`: Aktiviert die Nutzung von OpenID Connect. Damit sind weitere Parameter erforderlich
* `SPRING_SECURITY_OAUTH2_CLIENT_REGISTRATION_CUSTOM_CLIENT_NAME`: Name. Wird beim zusätzlichen Loginbutton angezeigt.
* `SPRING_SECURITY_OAUTH2_CLIENT_REGISTRATION_CUSTOM_CLIENT_ID`: Client-ID
* `SPRING_SECURITY_OAUTH2_CLIENT_REGISTRATION_CUSTOM_CLIENT_SECRET`: Client-Secret
* `SPRING_SECURITY_OAUTH2_CLIENT_REGISTRATION_CUSTOM_CLIENT_SCOPE[0]`: Hier sollte immer `openid` angegeben werden.
* `SPRING_SECURITY_OAUTH2_CLIENT_PROVIDER_CUSTOM_ISSUER_URI`: Die URI des Providers,
z.B. `https://auth.example.com/realm/example`
* `SPRING_SECURITY_OAUTH2_CLIENT_PROVIDER_CUSTOM_USER_NAME_ATTRIBUTE`: Name des Attributes, welches den Benutzernamen
enthält.
Oft verwendet: `preferred_username`
Ist die Nutzung von OpenID Connect konfiguriert, erscheint ein zusätzlicher Login-Button zur Nutzung mit OpenID Connect
und dem konfigurierten `CLIENT_NAME`.
![Login mit OpenID Connect](docs/login.png)
Weitere Informationen zur Konfiguration des OIDC-Providers
sind [hier](https://docs.spring.io/spring-security/reference/servlet/oauth2/index.html#oauth2-client)
zu finden.
#### Auswirkungen auf den dargestellten Inhalt
Nur Administratoren haben Zugriff auf den Konfigurationsbereich, nur angemeldete Benutzer können die anonymisierte oder
pseudonymisierte Patienten-ID einsehen.
pseudonymisierte Patienten-ID sowie den Qualitätsbericht des bwHC-Backends einsehen.
Wurde kein Administrator-Account konfiguriert, sind diese Inhalte generell nicht verfügbar.
### Tokenbasierte Authentifizierung für MTBFile-Endpunkt
Die Anwendung unterstützt das Erstellen und Nutzen einer tokenbasierten Authentifizierung für den MTB-File-Endpunkt.
Dies kann mit der Umgebungsvariable `APP_SECURITY_ENABLE_TOKENS` aktiviert (`true` oder `false`) werden
und ist als Standardeinstellung nicht aktiv.
Ist diese Einstellung aktiviert worden, ist es Administratoren möglich, Zugriffstokens für Onkostar zu erstellen, die
zur Nutzung des MTB-File-Endpunkts eine HTTP-Basic-Authentifizierung voraussetzen.
![Tokenverwaltung](docs/tokens.png)
In diesem Fall können den Endpunkt für das Onkostar-Plugin **[onkostar-plugin-dnpmexport](https://github.com/CCC-MF/onkostar-plugin-dnpmexport)** wie folgt konfigurieren:
```
https://testonkostar:MTg1NTL...NGU4@etl.example.com/mtbfile
```
Ist die Verwendung von Tokens aktiv, werden Anfragen ohne die Angabe der Token-Information abgelehnt.
### Transformation von Werten
In Onkostar kann es vorkommen, dass ein Wert eines Merkmalskatalogs an einem Standort angepasst wurde und dadurch nicht dem Wert entspricht,
@ -170,8 +225,37 @@ Diese Anwendung ist auch als Docker-Image verfügbar: https://github.com/CCC-MF/
./gradlew bootBuildImage
```
### Integration eines eigenen Root CA Zertifikats
Wird eine eigene Root CA verwendet, die nicht offiziell signiert ist, wird es zu Problemen beim SSL-Handshake kommen, wenn z.B. gPAS zur Generierung von Pseudonymen verwendet wird.
Hier bietet es sich an, das Root CA Zertifikat in das Image zu integrieren.
#### Integration beim Bauen des Images
Hier muss die Zeile `"BP_EMBED_CERTS" to "true"` in der Datei `build.gradle.kts` verwendet werden und darf nicht als Kommentar verwendet werden.
Die PEM-Datei mit dem/den Root CA Zertifikat(en) muss dabei im vorbereiteten Verzeichnis [`bindings/ca-certificates`](bindings/ca-certificates) enthalten sein.
#### Integration zur Laufzeit
Hier muss die Umgebungsvariable `SERVICE_BINDING_ROOT` z.B. auf den Wert `/bindings` gesetzt sein.
Zudem muss ein Verzeichnis `bindings/ca-certificates` - analog zum Verzeichnis [`bindings/ca-certificates`](bindings/ca-certificates) mit einer PEM-Datei als Docker-Volume eingebunden werden.
Beispiel für Docker-Compose:
```
...
environment:
SERVICE_BINDING_ROOT: /bindings
...
volumes:
- "/path/to/bindings/ca-certificates/:/bindings/ca-certificates/:ro"
...
```
## Deployment
*Ausführen als Docker Conatiner:*
*Ausführen als Docker Container:*
```bash
cd ./deploy

5
bindings/README.md Normal file
View File

@ -0,0 +1,5 @@
# Hinweis für Root CA Zertifikate
PEM-Datei(en) in das Verzeichnis `ca-certificates` ablegen.
Die Datei `type` gibt dabei an, dass hier CA Zertifikate zu finden sind.

View File

@ -0,0 +1 @@
ca-certificates

View File

@ -4,20 +4,23 @@ import org.springframework.boot.gradle.tasks.bundling.BootBuildImage
plugins {
war
id("org.springframework.boot") version "3.2.1"
id("org.springframework.boot") version "3.2.2"
id("io.spring.dependency-management") version "1.1.4"
kotlin("jvm") version "1.9.22"
kotlin("plugin.spring") version "1.9.22"
}
group = "de.ukw.ccc"
version = "0.4.0"
version = "0.7.0"
var versions = mapOf(
"bwhc-dto-java" to "0.2.0",
"hapi-fhir" to "6.10.2",
"httpclient5" to "5.2.1",
"mockito-kotlin" to "5.2.1"
"mockito-kotlin" to "5.2.1",
// Webjars
"echarts" to "5.4.3",
"htmx.org" to "1.9.10"
)
java {
@ -55,6 +58,7 @@ dependencies {
implementation("org.springframework.boot:spring-boot-starter-web")
implementation("org.springframework.boot:spring-boot-starter-data-jdbc")
implementation("org.springframework.boot:spring-boot-starter-security")
implementation("org.springframework.boot:spring-boot-starter-oauth2-client")
implementation("org.thymeleaf.extras:thymeleaf-extras-springsecurity6")
implementation("com.fasterxml.jackson.module:jackson-module-kotlin")
implementation("org.springframework.kafka:spring-kafka")
@ -66,6 +70,9 @@ dependencies {
implementation("ca.uhn.hapi.fhir:hapi-fhir-structures-r4:${versions["hapi-fhir"]}")
implementation("org.apache.httpcomponents.client5:httpclient5:${versions["httpclient5"]}")
implementation("com.jayway.jsonpath:json-path")
implementation("org.webjars:webjars-locator:0.50")
implementation("org.webjars.npm:echarts:${versions["echarts"]}")
implementation("org.webjars.npm:htmx.org:${versions["htmx.org"]}")
runtimeOnly("org.mariadb.jdbc:mariadb-java-client")
runtimeOnly("org.postgresql:postgresql")
developmentOnly("org.springframework.boot:spring-boot-devtools")
@ -73,6 +80,7 @@ dependencies {
annotationProcessor("org.springframework.boot:spring-boot-configuration-processor")
providedRuntime("org.springframework.boot:spring-boot-starter-tomcat")
testImplementation("org.springframework.boot:spring-boot-starter-test")
testImplementation("org.springframework.security:spring-security-test")
testImplementation("io.projectreactor:reactor-test")
testImplementation("org.mockito.kotlin:mockito-kotlin:${versions["mockito-kotlin"]}")
integrationTestImplementation("org.testcontainers:junit-jupiter")
@ -105,7 +113,14 @@ task<Test>("integrationTest") {
tasks.named<BootBuildImage>("bootBuildImage") {
imageName.set("ghcr.io/ccc-mf/etl-processor")
// Binding for CA Certs
bindings.set(listOf(
"$rootDir/bindings/ca-certificates/:/platform/bindings/ca-certificates"
))
environment.set(environment.get() + mapOf(
// Enable this line to embed CA Certs into image on build time
//"BP_EMBED_CERTS" to "true",
"BP_OCI_SOURCE" to "https://github.com/CCC-MF/etl-processor",
"BP_OCI_LICENSES" to "AGPLv3",
"BP_OCI_DESCRIPTION" to "ETL Processor for bwHC MTB files"

View File

@ -18,6 +18,8 @@ services:
APP_KAFKA_GROUP_ID: ${DNPM_KAFKA_GROUP_ID}
APP_KAFKA_RESPONSE_TOPIC: ${DNPM_KAFKA_RESPONSE_TOPIC}
APP_REST_URI: ${DNPM_BWHC_REST_URI}
APP_SECURITY_ADMIN_USER: ${DNPM_ADMIN_USER}
APP_SECURITY_ADMIN_PASSWORD: ${DNPM_ADMIN_PASSWORD}
SPRING_DATASOURCE_URL: ${DNPM_DATASOURCE_URL}
SPRING_DATASOURCE_PASSWORD: ${DNPM_MARIADB_USER_PW}
SPRING_DATASOURCE_USERNAME: ${DNPM_MARIADB_DB}

View File

@ -2,6 +2,10 @@
DNPM_MONITORING_HTTP_PORT=8088
DNPM_LOG_LEVEL=INFO
# ADMIN USER CREDENTIALS
DNPM_ADMIN_USER=admin
DNPM_ADMIN_PASSWORD=
# GPAS or BUILDIN
DNPM_PSEUDONYMIZE_GENERATOR=BUILDIN
DNPM_APP_PSEUDONYMIZE_PREFIX=ANONYM

BIN
docs/login.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 9.2 KiB

BIN
docs/tokens.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 30 KiB

View File

@ -48,7 +48,8 @@ import org.testcontainers.junit.jupiter.Testcontainers
@MockBean(MtbFileSender::class)
@TestPropertySource(
properties = [
"app.rest.uri=http://example.com"
"app.rest.uri=http://example.com",
"app.pseudonymize.generator=buildin"
]
)
class EtlProcessorApplicationTests : AbstractTestcontainerTest() {
@ -64,6 +65,7 @@ class EtlProcessorApplicationTests : AbstractTestcontainerTest() {
@AutoConfigureMockMvc
@TestPropertySource(
properties = [
"app.pseudonymize.generator=buildin",
"app.transformations[0].path=diagnoses[*].icd10.version",
"app.transformations[0].from=2013",
"app.transformations[0].to=2014",

View File

@ -1,7 +1,7 @@
/*
* This file is part of ETL-Processor
*
* Copyright (c) 2023 Comprehensive Cancer Center Mainfranken, Datenintegrationszentrum Philipps-Universität Marburg and Contributors
* Copyright (c) 2024 Comprehensive Cancer Center Mainfranken, Datenintegrationszentrum Philipps-Universität Marburg and Contributors
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
@ -23,6 +23,10 @@ import com.fasterxml.jackson.databind.ObjectMapper
import dev.dnpm.etl.processor.monitoring.RequestRepository
import dev.dnpm.etl.processor.output.KafkaMtbFileSender
import dev.dnpm.etl.processor.output.RestMtbFileSender
import dev.dnpm.etl.processor.pseudonym.AnonymizingGenerator
import dev.dnpm.etl.processor.pseudonym.GpasPseudonymGenerator
import dev.dnpm.etl.processor.services.TokenRepository
import dev.dnpm.etl.processor.services.TokenService
import org.assertj.core.api.Assertions.assertThat
import org.junit.jupiter.api.Nested
import org.junit.jupiter.api.Test
@ -31,13 +35,27 @@ import org.springframework.beans.factory.NoSuchBeanDefinitionException
import org.springframework.boot.autoconfigure.kafka.KafkaAutoConfiguration
import org.springframework.boot.test.context.SpringBootTest
import org.springframework.boot.test.mock.mockito.MockBean
import org.springframework.boot.test.mock.mockito.MockBeans
import org.springframework.context.ApplicationContext
import org.springframework.security.crypto.password.PasswordEncoder
import org.springframework.security.provisioning.InMemoryUserDetailsManager
import org.springframework.test.context.ContextConfiguration
import org.springframework.test.context.TestPropertySource
@SpringBootTest
@ContextConfiguration(classes = [AppConfiguration::class, KafkaAutoConfiguration::class, AppKafkaConfiguration::class, AppRestConfiguration::class])
@ContextConfiguration(classes = [
AppConfiguration::class,
AppSecurityConfiguration::class,
KafkaAutoConfiguration::class,
AppKafkaConfiguration::class,
AppRestConfiguration::class
])
@MockBean(ObjectMapper::class)
@TestPropertySource(
properties = [
"app.pseudonymize.generator=BUILDIN",
]
)
class AppConfigurationTest {
@Nested
@ -116,4 +134,108 @@ class AppConfigurationTest {
}
@Nested
inner class AppConfigurationPseudonymizeTest {
@Nested
@TestPropertySource(
properties = [
"app.pseudonymize.generator=",
"app.pseudonymizer=buildin",
]
)
inner class AppConfigurationPseudonymizerBuildinTest(private val context: ApplicationContext) {
@Test
fun shouldUseConfiguredGenerator() {
assertThat(context.getBean(AnonymizingGenerator::class.java)).isNotNull
}
}
@Nested
@TestPropertySource(
properties = [
"app.pseudonymize.generator=",
"app.pseudonymizer=gpas",
]
)
inner class AppConfigurationPseudonymizerGpasTest(private val context: ApplicationContext) {
@Test
fun shouldUseConfiguredGenerator() {
assertThat(context.getBean(GpasPseudonymGenerator::class.java)).isNotNull
}
}
@Nested
@TestPropertySource(
properties = [
"app.pseudonymize.generator=buildin",
"app.pseudonymizer=",
]
)
inner class AppConfigurationPseudonymizeGeneratorBuildinTest(private val context: ApplicationContext) {
@Test
fun shouldUseConfiguredGenerator() {
assertThat(context.getBean(AnonymizingGenerator::class.java)).isNotNull
}
}
@Nested
@TestPropertySource(
properties = [
"app.pseudonymize.generator=gpas",
"app.pseudonymizer=",
]
)
inner class AppConfigurationPseudonymizeGeneratorGpasTest(private val context: ApplicationContext) {
@Test
fun shouldUseConfiguredGenerator() {
assertThat(context.getBean(GpasPseudonymGenerator::class.java)).isNotNull
}
}
@Nested
@TestPropertySource(
properties = [
"app.security.enable-tokens=true"
]
)
@MockBeans(value = [
MockBean(InMemoryUserDetailsManager::class),
MockBean(PasswordEncoder::class),
MockBean(TokenRepository::class)
])
inner class AppConfigurationTokenEnabledTest(private val context: ApplicationContext) {
@Test
fun checkTokenService() {
assertThat(context.getBean(TokenService::class.java)).isNotNull
}
}
@Nested
@MockBeans(value = [
MockBean(InMemoryUserDetailsManager::class),
MockBean(PasswordEncoder::class),
MockBean(TokenRepository::class)
])
inner class AppConfigurationTokenDisabledTest(private val context: ApplicationContext) {
@Test
fun checkTokenService() {
assertThrows<NoSuchBeanDefinitionException> { context.getBean(TokenService::class.java) }
}
}
}
}

View File

@ -46,6 +46,7 @@ import java.util.*
@MockBean(MtbFileSender::class)
@TestPropertySource(
properties = [
"app.pseudonymize.generator=buildin",
"app.rest.uri=http://example.com"
]
)

View File

@ -0,0 +1,110 @@
/*
* This file is part of ETL-Processor
*
* Copyright (c) 2024 Comprehensive Cancer Center Mainfranken, Datenintegrationszentrum Philipps-Universität Marburg and Contributors
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
package dev.dnpm.etl.processor.web
import dev.dnpm.etl.processor.config.AppSecurityConfiguration
import dev.dnpm.etl.processor.monitoring.ConnectionCheckService
import dev.dnpm.etl.processor.output.MtbFileSender
import dev.dnpm.etl.processor.pseudonym.Generator
import dev.dnpm.etl.processor.services.RequestProcessor
import dev.dnpm.etl.processor.services.TransformationService
import org.junit.jupiter.api.BeforeEach
import org.junit.jupiter.api.Test
import org.junit.jupiter.api.extension.ExtendWith
import org.mockito.junit.jupiter.MockitoExtension
import org.springframework.beans.factory.annotation.Autowired
import org.springframework.boot.test.autoconfigure.web.servlet.WebMvcTest
import org.springframework.boot.test.mock.mockito.MockBean
import org.springframework.http.HttpHeaders
import org.springframework.http.MediaType
import org.springframework.security.test.web.servlet.request.SecurityMockMvcRequestPostProcessors.anonymous
import org.springframework.security.test.web.servlet.request.SecurityMockMvcRequestPostProcessors.user
import org.springframework.test.context.ContextConfiguration
import org.springframework.test.context.TestPropertySource
import org.springframework.test.context.junit.jupiter.SpringExtension
import org.springframework.test.web.servlet.MockMvc
import org.springframework.test.web.servlet.get
import reactor.core.publisher.Sinks
abstract class MockSink : Sinks.Many<Boolean>
@WebMvcTest(controllers = [ConfigController::class])
@ExtendWith(value = [MockitoExtension::class, SpringExtension::class])
@ContextConfiguration(
classes = [
ConfigController::class,
AppSecurityConfiguration::class
]
)
@TestPropertySource(
properties = [
"app.pseudonymize.generator=BUILDIN",
"app.security.admin-user=admin",
"app.security.admin-password={noop}very-secret",
"app.security.enable-tokens=true"
]
)
@MockBean(name = "configsUpdateProducer", classes = [MockSink::class])
@MockBean(
Generator::class,
MtbFileSender::class,
ConnectionCheckService::class,
RequestProcessor::class,
TransformationService::class
)
class ConfigControllerTest {
private lateinit var mockMvc: MockMvc
private lateinit var requestProcessor: RequestProcessor
@BeforeEach
fun setup(
@Autowired mockMvc: MockMvc,
@Autowired requestProcessor: RequestProcessor
) {
this.mockMvc = mockMvc
this.requestProcessor = requestProcessor
}
@Test
fun testShouldShowConfigPageIfLoggedIn() {
mockMvc.get("/configs") {
with(user("admin").roles("ADMIN"))
accept(MediaType.TEXT_HTML)
}.andExpect {
status { isOk() }
}
}
@Test
fun testShouldRedirectToLoginPageIfNotLoggedIn() {
mockMvc.get("/configs") {
with(anonymous())
accept(MediaType.TEXT_HTML)
}.andExpect {
status { isFound() }
header {
stringValues(HttpHeaders.LOCATION, "http://localhost/login")
}
}
}
}

View File

@ -0,0 +1,157 @@
/*
* This file is part of ETL-Processor
*
* Copyright (c) 2024 Comprehensive Cancer Center Mainfranken, Datenintegrationszentrum Philipps-Universität Marburg and Contributors
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
package dev.dnpm.etl.processor.web
import com.fasterxml.jackson.databind.ObjectMapper
import de.ukw.ccc.bwhc.dto.*
import dev.dnpm.etl.processor.config.AppSecurityConfiguration
import dev.dnpm.etl.processor.services.RequestProcessor
import dev.dnpm.etl.processor.services.TokenRepository
import org.junit.jupiter.api.BeforeEach
import org.junit.jupiter.api.Test
import org.junit.jupiter.api.extension.ExtendWith
import org.mockito.ArgumentMatchers.anyString
import org.mockito.junit.jupiter.MockitoExtension
import org.mockito.kotlin.any
import org.mockito.kotlin.never
import org.mockito.kotlin.times
import org.mockito.kotlin.verify
import org.springframework.beans.factory.annotation.Autowired
import org.springframework.boot.test.autoconfigure.web.servlet.WebMvcTest
import org.springframework.boot.test.mock.mockito.MockBean
import org.springframework.http.MediaType
import org.springframework.security.test.web.servlet.request.SecurityMockMvcRequestPostProcessors.anonymous
import org.springframework.security.test.web.servlet.request.SecurityMockMvcRequestPostProcessors.user
import org.springframework.test.context.ContextConfiguration
import org.springframework.test.context.TestPropertySource
import org.springframework.test.context.junit.jupiter.SpringExtension
import org.springframework.test.web.servlet.MockMvc
import org.springframework.test.web.servlet.delete
import org.springframework.test.web.servlet.post
@WebMvcTest(controllers = [MtbFileRestController::class])
@ExtendWith(value = [MockitoExtension::class, SpringExtension::class])
@ContextConfiguration(
classes = [
MtbFileRestController::class,
AppSecurityConfiguration::class
]
)
@MockBean(TokenRepository::class, RequestProcessor::class)
@TestPropertySource(
properties = [
"app.pseudonymize.generator=BUILDIN",
"app.security.admin-user=admin",
"app.security.admin-password={noop}very-secret",
"app.security.enable-tokens=true"
]
)
class MtbFileRestControllerTest {
private lateinit var mockMvc: MockMvc
private lateinit var requestProcessor: RequestProcessor
@BeforeEach
fun setup(
@Autowired mockMvc: MockMvc,
@Autowired requestProcessor: RequestProcessor
) {
this.mockMvc = mockMvc
this.requestProcessor = requestProcessor
}
@Test
fun testShouldGrantPermissionToSendMtbFile() {
mockMvc.post("/mtbfile") {
with(user("onkostarserver").roles("MTBFILE"))
contentType = MediaType.APPLICATION_JSON
content = ObjectMapper().writeValueAsString(mtbFile)
}.andExpect {
status { isAccepted() }
}
verify(requestProcessor, times(1)).processMtbFile(any())
}
@Test
fun testShouldDenyPermissionToSendMtbFile() {
mockMvc.post("/mtbfile") {
with(anonymous())
contentType = MediaType.APPLICATION_JSON
content = ObjectMapper().writeValueAsString(mtbFile)
}.andExpect {
status { isUnauthorized() }
}
verify(requestProcessor, never()).processMtbFile(any())
}
@Test
fun testShouldGrantPermissionToDeletePatientData() {
mockMvc.delete("/mtbfile/12345678") {
with(user("onkostarserver").roles("MTBFILE"))
}.andExpect {
status { isAccepted() }
}
verify(requestProcessor, times(1)).processDeletion(anyString())
}
@Test
fun testShouldDenyPermissionToDeletePatientData() {
mockMvc.delete("/mtbfile/12345678") {
with(anonymous())
}.andExpect {
status { isUnauthorized() }
}
verify(requestProcessor, never()).processDeletion(anyString())
}
companion object {
val mtbFile: MtbFile = MtbFile.builder()
.withPatient(
Patient.builder()
.withId("PID")
.withBirthDate("2000-08-08")
.withGender(Patient.Gender.MALE)
.build()
)
.withConsent(
Consent.builder()
.withId("1")
.withStatus(Consent.Status.ACTIVE)
.withPatient("PID")
.build()
)
.withEpisode(
Episode.builder()
.withId("1")
.withPatient("PID")
.withPeriod(PeriodStart("2023-08-08"))
.build()
)
.build()
}
}

View File

@ -20,11 +20,16 @@
package dev.dnpm.etl.processor.config
import org.springframework.boot.context.properties.ConfigurationProperties
import org.springframework.boot.context.properties.DeprecatedConfigurationProperty
@ConfigurationProperties(AppConfigProperties.NAME)
data class AppConfigProperties(
var bwhcUri: String?,
var generator: PseudonymGenerator = PseudonymGenerator.BUILDIN,
@get:DeprecatedConfigurationProperty(
reason = "Deprecated in favor of 'app.pseudonymize.generator'",
replacement = "app.pseudonymize.generator"
)
var pseudonymizer: PseudonymGenerator = PseudonymGenerator.BUILDIN,
var transformations: List<TransformationProperties> = listOf()
) {
companion object {
@ -34,6 +39,7 @@ data class AppConfigProperties(
@ConfigurationProperties(PseudonymizeConfigProperties.NAME)
data class PseudonymizeConfigProperties(
var generator: PseudonymGenerator = PseudonymGenerator.BUILDIN,
val prefix: String = "UNKNOWN",
) {
companion object {
@ -80,6 +86,8 @@ data class KafkaTargetProperties(
data class SecurityConfigProperties(
val adminUser: String?,
val adminPassword: String?,
val enableTokens: Boolean = false,
val enableOidc: Boolean = false
) {
companion object {
const val NAME = "app.security"

View File

@ -25,9 +25,12 @@ import dev.dnpm.etl.processor.pseudonym.AnonymizingGenerator
import dev.dnpm.etl.processor.pseudonym.Generator
import dev.dnpm.etl.processor.pseudonym.GpasPseudonymGenerator
import dev.dnpm.etl.processor.pseudonym.PseudonymizeService
import dev.dnpm.etl.processor.services.TokenRepository
import dev.dnpm.etl.processor.services.TokenService
import dev.dnpm.etl.processor.services.Transformation
import dev.dnpm.etl.processor.services.TransformationService
import org.slf4j.LoggerFactory
import org.springframework.boot.autoconfigure.condition.ConditionalOnMissingBean
import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty
import org.springframework.boot.context.properties.EnableConfigurationProperties
import org.springframework.context.annotation.Bean
@ -36,6 +39,9 @@ import org.springframework.retry.policy.SimpleRetryPolicy
import org.springframework.retry.support.RetryTemplate
import org.springframework.retry.support.RetryTemplateBuilder
import org.springframework.scheduling.annotation.EnableScheduling
import org.springframework.security.crypto.password.PasswordEncoder
import org.springframework.security.provisioning.InMemoryUserDetailsManager
import org.springframework.security.provisioning.UserDetailsManager
import reactor.core.publisher.Sinks
import kotlin.time.Duration.Companion.seconds
import kotlin.time.toJavaDuration
@ -54,18 +60,32 @@ class AppConfiguration {
private val logger = LoggerFactory.getLogger(AppConfiguration::class.java)
@ConditionalOnProperty(value = ["app.pseudonymizer"], havingValue = "GPAS")
@ConditionalOnProperty(value = ["app.pseudonymize.generator"], havingValue = "GPAS")
@Bean
fun gpasPseudonymGenerator(configProperties: GPasConfigProperties): Generator {
return GpasPseudonymGenerator(configProperties)
}
@ConditionalOnProperty(value = ["app.pseudonymizer"], havingValue = "BUILDIN", matchIfMissing = true)
@ConditionalOnProperty(value = ["app.pseudonymize.generator"], havingValue = "BUILDIN", matchIfMissing = true)
@Bean
fun buildinPseudonymGenerator(): Generator {
return AnonymizingGenerator()
}
@ConditionalOnProperty(value = ["app.pseudonymizer"], havingValue = "GPAS")
@ConditionalOnMissingBean
@Bean
fun gpasPseudonymGeneratorOnDeprecatedProperty(configProperties: GPasConfigProperties): Generator {
return GpasPseudonymGenerator(configProperties)
}
@ConditionalOnProperty(value = ["app.pseudonymizer"], havingValue = "BUILDIN")
@ConditionalOnMissingBean
@Bean
fun buildinPseudonymGeneratorOnDeprecatedProperty(): Generator {
return AnonymizingGenerator()
}
@Bean
fun pseudonymizeService(
generator: Generator,
@ -79,11 +99,6 @@ class AppConfiguration {
return ReportService(objectMapper)
}
@Bean
fun statisticsUpdateProducer(): Sinks.Many<Any> {
return Sinks.many().multicast().directBestEffort()
}
@Bean
fun transformationService(
objectMapper: ObjectMapper,
@ -104,5 +119,21 @@ class AppConfiguration {
.build()
}
@ConditionalOnProperty(value = ["app.security.enable-tokens"], havingValue = "true")
@Bean
fun tokenService(userDetailsManager: InMemoryUserDetailsManager, passwordEncoder: PasswordEncoder, tokenRepository: TokenRepository): TokenService {
return TokenService(userDetailsManager, passwordEncoder, tokenRepository)
}
@Bean
fun statisticsUpdateProducer(): Sinks.Many<Any> {
return Sinks.many().multicast().directBestEffort()
}
@Bean
fun configsUpdateProducer(): Sinks.Many<Boolean> {
return Sinks.many().multicast().directBestEffort()
}
}

View File

@ -38,6 +38,7 @@ import org.springframework.kafka.core.KafkaTemplate
import org.springframework.kafka.listener.ContainerProperties
import org.springframework.kafka.listener.KafkaMessageListenerContainer
import org.springframework.retry.support.RetryTemplate
import reactor.core.publisher.Sinks
@Configuration
@EnableConfigurationProperties(
@ -81,8 +82,8 @@ class AppKafkaConfiguration {
}
@Bean
fun connectionCheckService(consumerFactory: ConsumerFactory<String, String>): ConnectionCheckService {
return KafkaConnectionCheckService(consumerFactory.createConsumer())
fun connectionCheckService(consumerFactory: ConsumerFactory<String, String>, configsUpdateProducer: Sinks.Many<Boolean>): ConnectionCheckService {
return KafkaConnectionCheckService(consumerFactory.createConsumer(), configsUpdateProducer)
}
}

View File

@ -32,6 +32,7 @@ import org.springframework.context.annotation.Configuration
import org.springframework.core.annotation.Order
import org.springframework.retry.support.RetryTemplate
import org.springframework.web.client.RestTemplate
import reactor.core.publisher.Sinks
@Configuration
@EnableConfigurationProperties(
@ -64,9 +65,10 @@ class AppRestConfiguration {
@Bean
fun connectionCheckService(
restTemplate: RestTemplate,
restTargetProperties: RestTargetProperties
restTargetProperties: RestTargetProperties,
configsUpdateProducer: Sinks.Many<Boolean>
): ConnectionCheckService {
return RestConnectionCheckService(restTemplate, restTargetProperties)
return RestConnectionCheckService(restTemplate, restTargetProperties, configsUpdateProducer)
}
}

View File

@ -76,12 +76,42 @@ class AppSecurityConfiguration(
}
@Bean
fun filterChain(http: HttpSecurity): SecurityFilterChain {
@ConditionalOnProperty(value = ["app.security.enable-oidc"], havingValue = "true")
fun filterChainOidc(http: HttpSecurity, passwordEncoder: PasswordEncoder): SecurityFilterChain {
http {
authorizeRequests {
authorize("/configs/**", hasRole("ADMIN"))
authorize("/mtbfile/**", hasAnyRole("MTBFILE"))
authorize("/report/**", fullyAuthenticated)
authorize(anyRequest, permitAll)
}
httpBasic {
realmName = "ETL-Processor"
}
formLogin {
loginPage = "/login"
}
oauth2Login {
loginPage = "/login"
}
csrf { disable() }
}
return http.build()
}
@Bean
@ConditionalOnProperty(value = ["app.security.enable-oidc"], havingValue = "false", matchIfMissing = true)
fun filterChain(http: HttpSecurity, passwordEncoder: PasswordEncoder): SecurityFilterChain {
http {
authorizeRequests {
authorize("/configs/**", hasRole("ADMIN"))
authorize("/mtbfile/**", hasAnyRole("MTBFILE"))
authorize("/report/**", hasRole("ADMIN"))
authorize(anyRequest, permitAll)
}
httpBasic {
realmName = "ETL-Processor"
}
formLogin {
loginPage = "/login"
}

View File

@ -24,9 +24,11 @@ import dev.dnpm.etl.processor.config.RestTargetProperties
import jakarta.annotation.PostConstruct
import org.apache.kafka.clients.consumer.Consumer
import org.apache.kafka.common.errors.TimeoutException
import org.springframework.beans.factory.annotation.Qualifier
import org.springframework.http.HttpStatus
import org.springframework.scheduling.annotation.Scheduled
import org.springframework.web.client.RestTemplate
import reactor.core.publisher.Sinks
import kotlin.time.Duration.Companion.seconds
import kotlin.time.toJavaDuration
@ -37,7 +39,9 @@ interface ConnectionCheckService {
}
class KafkaConnectionCheckService(
private val consumer: Consumer<String, String>
private val consumer: Consumer<String, String>,
@Qualifier("configsUpdateProducer")
private val configsUpdateProducer: Sinks.Many<Boolean>
) : ConnectionCheckService {
private var connectionAvailable: Boolean = false
@ -51,6 +55,7 @@ class KafkaConnectionCheckService(
} catch (e: TimeoutException) {
false
}
configsUpdateProducer.emitNext(connectionAvailable, Sinks.EmitFailureHandler.FAIL_FAST)
}
override fun connectionAvailable(): Boolean {
@ -61,7 +66,9 @@ class KafkaConnectionCheckService(
class RestConnectionCheckService(
private val restTemplate: RestTemplate,
private val restTargetProperties: RestTargetProperties
private val restTargetProperties: RestTargetProperties,
@Qualifier("configsUpdateProducer")
private val configsUpdateProducer: Sinks.Many<Boolean>
) : ConnectionCheckService {
private var connectionAvailable: Boolean = false
@ -77,6 +84,7 @@ class RestConnectionCheckService(
} catch (e: Exception) {
false
}
configsUpdateProducer.emitNext(connectionAvailable, Sinks.EmitFailureHandler.FAIL_FAST)
}
override fun connectionAvailable(): Boolean {

View File

@ -20,6 +20,8 @@
package dev.dnpm.etl.processor.monitoring
import org.springframework.data.annotation.Id
import org.springframework.data.domain.Page
import org.springframework.data.domain.Pageable
import org.springframework.data.jdbc.repository.query.Query
import org.springframework.data.relational.core.mapping.Embedded
import org.springframework.data.relational.core.mapping.Table
@ -59,6 +61,8 @@ interface RequestRepository : CrudRepository<Request, Long>, PagingAndSortingRep
fun findByUuidEquals(uuid: String): Optional<Request>
fun findRequestByPatientId(patientId: String, pageable: Pageable): Page<Request>
@Query("SELECT count(*) AS count, status FROM request WHERE type = 'MTB_FILE' GROUP BY status ORDER BY status, count DESC;")
fun countStates(): List<CountedState>

View File

@ -0,0 +1,92 @@
/*
* This file is part of ETL-Processor
*
* Copyright (c) 2024 Comprehensive Cancer Center Mainfranken, Datenintegrationszentrum Philipps-Universität Marburg and Contributors
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
package dev.dnpm.etl.processor.services
import jakarta.annotation.PostConstruct
import org.springframework.data.annotation.Id
import org.springframework.data.relational.core.mapping.Table
import org.springframework.data.repository.CrudRepository
import org.springframework.data.repository.findByIdOrNull
import org.springframework.security.core.userdetails.User
import org.springframework.security.crypto.password.PasswordEncoder
import org.springframework.security.provisioning.InMemoryUserDetailsManager
import java.time.Instant
import java.util.*
class TokenService(
private val userDetailsManager: InMemoryUserDetailsManager,
private val passwordEncoder: PasswordEncoder,
private val tokenRepository: TokenRepository
) {
@PostConstruct
fun setup() {
tokenRepository.findAll().forEach {
userDetailsManager.createUser(
User.withUsername(it.username)
.password(it.password)
.roles("MTBFILE")
.build()
)
}
}
fun addToken(name: String): Result<String> {
val username = name.lowercase().replace("""[^a-z0-9]""".toRegex(), "")
if (userDetailsManager.userExists(username)) {
return Result.failure(RuntimeException("Cannot use token name"))
}
val password = Base64.getEncoder().encodeToString(UUID.randomUUID().toString().encodeToByteArray())
val encodedPassword = passwordEncoder.encode(password).toString()
userDetailsManager.createUser(
User.withUsername(username)
.password(encodedPassword)
.roles("MTBFILE")
.build()
)
tokenRepository.save(Token(name = name, username = username, password = encodedPassword))
return Result.success("$username:$password")
}
fun deleteToken(id: Long) {
val token = tokenRepository.findByIdOrNull(id) ?: return
userDetailsManager.deleteUser(token.username)
tokenRepository.delete(token)
}
fun findAll(): List<Token> {
return tokenRepository.findAll().toList()
}
}
@Table("token")
data class Token(
@Id val id: Long? = null,
val name: String,
val username: String,
val password: String,
val createdAt: Instant = Instant.now()
)
interface TokenRepository : CrudRepository<Token, Long>

View File

@ -22,20 +22,28 @@ package dev.dnpm.etl.processor.web
import dev.dnpm.etl.processor.monitoring.ConnectionCheckService
import dev.dnpm.etl.processor.output.MtbFileSender
import dev.dnpm.etl.processor.pseudonym.Generator
import dev.dnpm.etl.processor.services.Token
import dev.dnpm.etl.processor.services.TokenService
import dev.dnpm.etl.processor.services.TransformationService
import org.springframework.beans.factory.annotation.Qualifier
import org.springframework.http.MediaType
import org.springframework.http.codec.ServerSentEvent
import org.springframework.stereotype.Controller
import org.springframework.ui.Model
import org.springframework.web.bind.annotation.GetMapping
import org.springframework.web.bind.annotation.RequestMapping
import org.springframework.web.bind.annotation.*
import reactor.core.publisher.Flux
import reactor.core.publisher.Sinks
@Controller
@RequestMapping(path = ["configs"])
class ConfigController(
@Qualifier("configsUpdateProducer")
private val configsUpdateProducer: Sinks.Many<Boolean>,
private val transformationService: TransformationService,
private val pseudonymGenerator: Generator,
private val mtbFileSender: MtbFileSender,
private val connectionCheckService: ConnectionCheckService
private val connectionCheckService: ConnectionCheckService,
private val tokenService: TokenService?
) {
@GetMapping
@ -44,9 +52,73 @@ class ConfigController(
model.addAttribute("mtbFileSender", mtbFileSender.javaClass.simpleName)
model.addAttribute("mtbFileEndpoint", mtbFileSender.endpoint())
model.addAttribute("connectionAvailable", connectionCheckService.connectionAvailable())
model.addAttribute("tokensEnabled", tokenService != null)
if (tokenService != null) {
model.addAttribute("tokens", tokenService.findAll())
} else {
model.addAttribute("tokens", listOf<Token>())
}
model.addAttribute("transformations", transformationService.getTransformations())
return "configs"
}
@GetMapping(params = ["connectionAvailable"])
fun connectionAvailable(model: Model): String {
model.addAttribute("mtbFileSender", mtbFileSender.javaClass.simpleName)
model.addAttribute("mtbFileEndpoint", mtbFileSender.endpoint())
model.addAttribute("connectionAvailable", connectionCheckService.connectionAvailable())
if (tokenService != null) {
model.addAttribute("tokensEnabled", true)
model.addAttribute("tokens", tokenService.findAll())
} else {
model.addAttribute("tokens", listOf<Token>())
}
return "configs/connectionAvailable"
}
@PostMapping(path = ["tokens"])
fun addToken(@ModelAttribute("name") name: String, model: Model): String {
if (tokenService == null) {
model.addAttribute("tokensEnabled", false)
model.addAttribute("success", false)
} else {
model.addAttribute("tokensEnabled", true)
val result = tokenService.addToken(name)
if (result.isSuccess) {
model.addAttribute("newTokenValue", result.getOrDefault(""))
model.addAttribute("success", true)
} else {
model.addAttribute("success", false)
}
model.addAttribute("tokens", tokenService.findAll())
}
return "configs/tokens"
}
@DeleteMapping(path = ["tokens/{id}"])
fun deleteToken(@PathVariable id: Long, model: Model): String {
if (tokenService != null) {
tokenService.deleteToken(id)
model.addAttribute("tokensEnabled", true)
model.addAttribute("tokens", tokenService.findAll())
} else {
model.addAttribute("tokensEnabled", false)
model.addAttribute("tokens", listOf<Token>())
}
return "configs/tokens"
}
@GetMapping(path = ["events"], produces = [MediaType.TEXT_EVENT_STREAM_VALUE])
fun events(): Flux<ServerSentEvent<Any>> {
return configsUpdateProducer.asFlux().map {
ServerSentEvent.builder<Any>()
.event("connection-available").id("none").data("")
.build()
}
}
}

View File

@ -40,13 +40,29 @@ class HomeController(
) {
@GetMapping
fun index(@PageableDefault(page = 0, size = 20, sort = ["processedAt"], direction = Sort.Direction.DESC) pageable: Pageable, model: Model): String {
fun index(
@PageableDefault(page = 0, size = 20, sort = ["processedAt"], direction = Sort.Direction.DESC) pageable: Pageable,
model: Model
): String {
val requests = requestRepository.findAll(pageable)
model.addAttribute("requests", requests)
return "index"
}
@GetMapping(path = ["patient/{patientId}"])
fun byPatient(
@PathVariable patientId: String,
@PageableDefault(page = 0, size = 20, sort = ["processedAt"], direction = Sort.Direction.DESC) pageable: Pageable,
model: Model
): String {
val requests = requestRepository.findRequestByPatientId(patientId, pageable)
model.addAttribute("patientId", patientId)
model.addAttribute("requests", requests)
return "index"
}
@GetMapping(path = ["/report/{id}"])
fun report(@PathVariable id: RequestId, model: Model): String {
val request = requestRepository.findByUuidEquals(id.toString()).orElse(null) ?: throw NotFoundException()

View File

@ -19,14 +19,28 @@
package dev.dnpm.etl.processor.web
import dev.dnpm.etl.processor.config.SecurityConfigProperties
import org.springframework.boot.autoconfigure.security.oauth2.client.OAuth2ClientProperties
import org.springframework.stereotype.Controller
import org.springframework.ui.Model
import org.springframework.web.bind.annotation.GetMapping
@Controller
class LoginController {
class LoginController(
private val securityConfigProperties: SecurityConfigProperties?,
private val oAuth2ClientProperties: OAuth2ClientProperties?
) {
@GetMapping(path = ["/login"])
fun login(): String {
fun login(model: Model): String {
if (securityConfigProperties?.enableOidc == true) {
model.addAttribute(
"oidcLogins",
oAuth2ClientProperties?.registration?.map { (key, value) -> Pair(key, value.clientName) }.orEmpty()
)
} else {
model.addAttribute("oidcLogins", emptyList<Pair<String, String>>())
}
return "login"
}

View File

@ -27,13 +27,19 @@ import org.springframework.http.ResponseEntity
import org.springframework.web.bind.annotation.*
@RestController
@RequestMapping(path = ["mtbfile"])
class MtbFileRestController(
private val requestProcessor: RequestProcessor,
) {
private val logger = LoggerFactory.getLogger(MtbFileRestController::class.java)
@PostMapping(path = ["/mtbfile"])
@GetMapping
fun info(): ResponseEntity<String> {
return ResponseEntity.ok("Test")
}
@PostMapping
fun mtbFile(@RequestBody mtbFile: MtbFile): ResponseEntity<Void> {
if (mtbFile.consent.status == Consent.Status.ACTIVE) {
logger.debug("Accepted MTB File for processing")
@ -45,7 +51,7 @@ class MtbFileRestController(
return ResponseEntity.accepted().build()
}
@DeleteMapping(path = ["/mtbfile/{patientId}"])
@DeleteMapping(path = ["{patientId}"])
fun deleteData(@PathVariable patientId: String): ResponseEntity<Void> {
logger.debug("Accepted patient ID to process deletion")
requestProcessor.processDeletion(patientId)

View File

@ -22,6 +22,7 @@ package dev.dnpm.etl.processor.web
import dev.dnpm.etl.processor.monitoring.RequestRepository
import dev.dnpm.etl.processor.monitoring.RequestStatus
import dev.dnpm.etl.processor.monitoring.RequestType
import org.springframework.beans.factory.annotation.Qualifier
import org.springframework.http.MediaType
import org.springframework.http.codec.ServerSentEvent
import org.springframework.web.bind.annotation.GetMapping
@ -38,6 +39,7 @@ import java.time.temporal.ChronoUnit
@RestController
@RequestMapping(path = ["/statistics"])
class StatisticsRestController(
@Qualifier("statisticsUpdateProducer")
private val statisticsUpdateProducer: Sinks.Many<Any>,
private val requestRepository: RequestRepository
) {
@ -132,6 +134,7 @@ class StatisticsRestController(
@GetMapping(path = ["events"], produces = [MediaType.TEXT_EVENT_STREAM_VALUE])
fun updater(): Flux<ServerSentEvent<Any>> {
return statisticsUpdateProducer.asFlux().flatMap {
println(it)
Flux.fromIterable(
listOf(
ServerSentEvent.builder<Any>()
@ -152,6 +155,10 @@ class StatisticsRestController(
.build(),
ServerSentEvent.builder<Any>()
.event("deleterequestpatientstates").id("none").data(this.requestPatientStates(true))
.build(),
ServerSentEvent.builder<Any>()
.event("newrequest").id("none").data("newrequest")
.build()
)
)

View File

@ -0,0 +1,8 @@
CREATE TABLE IF NOT EXISTS token
(
id int auto_increment primary key,
name varchar(255) not null,
username varchar(255) not null unique,
password varchar(255) not null,
created_at datetime default utc_timestamp() not null
);

View File

@ -0,0 +1,9 @@
CREATE TABLE IF NOT EXISTS token
(
id serial,
name varchar(255) not null,
username varchar(255) not null unique,
password varchar(255) not null,
created_at timestamp with time zone default now() not null,
PRIMARY KEY (id)
);

File diff suppressed because one or more lines are too long

View File

@ -4,14 +4,17 @@ const dateFormat = new Intl.DateTimeFormat('de-DE', dateFormatOptions);
const dateTimeFormatOptions = { year: 'numeric', month: '2-digit', day: '2-digit', hour: '2-digit', minute: 'numeric', second: 'numeric' };
const dateTimeFormat = new Intl.DateTimeFormat('de-DE', dateTimeFormatOptions);
window.addEventListener('load', () => {
const formatTimeElements = () => {
Array.from(document.getElementsByTagName('time')).forEach((timeTag) => {
let date = Date.parse(timeTag.getAttribute('datetime'));
if (! isNaN(date)) {
timeTag.innerText = dateTimeFormat.format(date);
}
});
});
};
window.addEventListener('load', formatTimeElements);
window.addEventListener('htmx:afterRequest', formatTimeElements);
function drawPieChart(url, elemId, title, data) {
if (data) {

View File

@ -202,13 +202,22 @@ form.samplecode-input input:focus-visible {
background: none;
}
.login-form form * {
.login-form form *,
.token-form form * {
padding: 0.5em;
border: 1px solid var(--table-border);
border-radius: 3px;
}
.login-form button {
.login-form form hr,
.token-form form hr {
padding: 0;
width: 100%;
}
.login-form button,
.login-form a.btn,
.token-form button {
margin: 1em 0;
background: var(--bg-blue);
color: white;
@ -521,3 +530,36 @@ input.inline:focus-visible {
.notification.error {
color: var(--bg-red);
}
a.reload {
display: none;
position: absolute;
height: 1.2em;
width: 1.2em;
background: var(--bg-red);
border-radius: 50%;
color: white;
text-decoration: none;
font-size: .6em;
align-content: center;
justify-content: center;
}
.new-token {
padding: 1em;
background: var(--bg-green-op);
}
.new-token > pre {
margin: 0;
border: 1px solid var(--bg-green);
padding: .5em;
width: max-content;
display: inline-block;
}
.no-token {
padding: 1em;
background: var(--bg-red-op);
}

View File

@ -37,22 +37,11 @@
</table>
</section>
<section>
<h2><span th:if="${connectionAvailable}"></span><span th:if="${not(connectionAvailable)}"></span> Verbindung zum bwHC-Backend</h2>
<div>
Verbindung über <code>[[ ${mtbFileSender} ]]</code>. Die Verbindung ist aktuell
<strong th:if="${connectionAvailable}" style="color: green">verfügbar.</strong>
<strong th:if="${not(connectionAvailable)}" style="color: red">nicht verfügbar.</strong>
</div>
<div class="connection-display border">
<img th:src="@{/server.png}" alt="ETL-Processor" />
<span class="connection" th:styleappend="${connectionAvailable ? 'available' : ''}"></span>
<img th:if="${mtbFileSender.startsWith('Rest')}" th:src="@{/server.png}" alt="bwHC-Backend" />
<img th:if="${mtbFileSender.startsWith('Kafka')}" th:src="@{/kafka.png}" alt="Kafka-Broker" />
<span>ETL-Processor</span>
<span></span>
<span th:if="${mtbFileSender.startsWith('Rest')}">bwHC-Backend</span>
<span th:if="${mtbFileSender.startsWith('Kafka')}">Kafka-Broker</span>
<section th:insert="~{configs/tokens.html}">
</section>
<section hx-ext="sse" th:sse-connect="@{/configs/events}">
<div th:insert="~{configs/connectionAvailable.html}" th:hx-get="@{/configs?connectionAvailable}" hx-trigger="sse:connection-available">
</div>
</section>
@ -100,5 +89,8 @@
</th:block>
</section>
</main>
<script th:src="@{/scripts.js}"></script>
<script th:src="@{/webjars/htmx.org/dist/htmx.min.js}"></script>
<script th:src="@{/webjars/htmx.org/dist/ext/sse.js}"></script>
</body>
</html>

View File

@ -0,0 +1,16 @@
<h2><span th:if="${connectionAvailable}"></span><span th:if="${not(connectionAvailable)}"></span> Verbindung zum bwHC-Backend</h2>
<div>
Verbindung über <code>[[ ${mtbFileSender} ]]</code>. Die Verbindung ist aktuell
<strong th:if="${connectionAvailable}" style="color: green">verfügbar.</strong>
<strong th:if="${not(connectionAvailable)}" style="color: red">nicht verfügbar.</strong>
</div>
<div class="connection-display border">
<img th:src="@{/server.png}" alt="ETL-Processor" />
<span class="connection" th:classappend="${connectionAvailable ? 'available' : ''}"></span>
<img th:if="${mtbFileSender.startsWith('Rest')}" th:src="@{/server.png}" alt="bwHC-Backend" />
<img th:if="${mtbFileSender.startsWith('Kafka')}" th:src="@{/kafka.png}" alt="Kafka-Broker" />
<span>ETL-Processor</span>
<span></span>
<span th:if="${mtbFileSender.startsWith('Rest')}">bwHC-Backend</span>
<span th:if="${mtbFileSender.startsWith('Kafka')}">Kafka-Broker</span>
</div>

View File

@ -0,0 +1,39 @@
<div th:if="${not tokensEnabled}">
<h2><span></span> Tokens</h2>
<p>Die Verwendung von Tokens ist nicht aktiviert.</p>
</div>
<div id="tokens" th:if="${tokensEnabled}">
<h2><span></span> Tokens</h2>
<div class="border">
<div th:if="${tokens.isEmpty()}">Noch keine Tokens vorhanden.</div>
<table th:if="${not tokens.isEmpty()}">
<thead>
<tr>
<th>Name</th>
<th>Erstellt</th>
<th></th>
</tr>
</thead>
<tbody>
<tr th:each="token : ${tokens}">
<td>[[ ${token.name} ]]</td>
<td><time th:datetime="${token.createdAt}">[[ ${token.createdAt} ]]</time></td>
<td><button class="btn btn-red" th:hx-delete="@{/configs/tokens/{id}(id=${token.id})}" hx-target="#tokens">Löschen</button></td>
</tr>
</tbody>
</table>
<div th:if="${newTokenValue != null and success}" class="new-token">
Verwendung über HTTP-Basic. Bitte notieren, wird nicht erneut angezeigt: <pre>[[ ${newTokenValue} ]]</pre>
</div>
<div th:if="${success != null and not success}" class="no-token">
Das Token konnte nicht erzeugt werden. Versuchen Sie einen anderen Namen.
</div>
<div class="token-form">
<form th:hx-post="@{/configs/tokens}" hx-target="#tokens">
<input placeholder="Token-Name" name="name" required />
<button>Token Erstellen</button>
</form>
</div>
</div>
</div>

View File

@ -21,6 +21,11 @@
<a th:href="@{/login}">Login</a>
</li>
<li class="login" sec:authorize="isAuthenticated()">
<span>
<span>👤</span>
<span sec:authentication="name">?</span>
</span>
&nbsp;
<a th:href="@{/logout}">Abmelden</a>
</li>
</ul>

View File

@ -9,16 +9,30 @@
<div th:replace="~{fragments.html :: nav}"></div>
<main>
<h1>Letzte Anfragen</h1>
<h1>Alle Anfragen<a id="reload-notify" class="reload" title="Neue Anfragen" th:href="@{/}"></a></h1>
<div>
<h2 th:if="${patientId != null}">
Betreffend Patienten-Pseudonym <span class="monospace" th:text="${patientId}">***</span>
<a class="btn btn-blue" th:if="${patientId != null}" th:href="@{/}">Alle anzeigen</a>
</h2>
</div>
<div class="border">
<div class="page-control">
<div th:if="${patientId == null}" class="page-control">
<a id="first-page-link" th:href="@{/(page=${0})}" title="Zum Anfang: Taste W" th:if="${not requests.isFirst()}">&larrb;</a><a th:if="${requests.isFirst()}">&larrb;</a>
<a id="prev-page-link" th:href="@{/(page=${requests.getNumber() - 1})}" title="Seite zurück: Taste A" th:if="${not requests.isFirst()}">&larr;</a><a th:if="${requests.isFirst()}">&larr;</a>
<span>Seite [[ ${requests.getNumber() + 1} ]] von [[ ${requests.getTotalPages()} ]]</span>
<a id="next-page-link" th:href="@{/(page=${requests.getNumber() + 1})}" title="Seite vor: Taste D" th:if="${not requests.isLast()}">&rarr;</a><a th:if="${requests.isLast()}">&rarr;</a>
<a id="last-page-link" th:href="@{/(page=${requests.getTotalPages() - 1})}" title="Zum Ende: Taste S" th:if="${not requests.isLast()}">&rarrb;</a><a th:if="${requests.isLast()}">&rarrb;</a>
</div>
<div th:if="${patientId != null}" class="page-control">
<a id="first-page-link" th:href="@{/patient/{patientId}(patientId=${patientId},page=${0})}" title="Zum Anfang: Taste W" th:if="${not requests.isFirst()}">&larrb;</a><a th:if="${requests.isFirst()}">&larrb;</a>
<a id="prev-page-link" th:href="@{/patient/{patientId}(patientId=${patientId},page=${requests.getNumber() - 1})}" title="Seite zurück: Taste A" th:if="${not requests.isFirst()}">&larr;</a><a th:if="${requests.isFirst()}">&larr;</a>
<span>Seite [[ ${requests.getNumber() + 1} ]] von [[ ${requests.getTotalPages()} ]]</span>
<a id="next-page-link" th:href="@{/patient/{patientId}(patientId=${patientId},page=${requests.getNumber() + 1})}" title="Seite vor: Taste D" th:if="${not requests.isLast()}">&rarr;</a><a th:if="${requests.isLast()}">&rarr;</a>
<a id="last-page-link" th:href="@{/patient/{patientId}(patientId=${patientId},page=${requests.getTotalPages() - 1})}" title="Zum Ende: Taste S" th:if="${not requests.isLast()}">&rarrb;</a><a th:if="${requests.isLast()}">&rarrb;</a>
</div>
<table class="paged">
<thead>
<tr>
@ -39,10 +53,16 @@
<td th:style="${request.type.value == 'delete'} ? 'color: red;'"><small>[[ ${request.type} ]]</small></td>
<td th:if="not ${request.report}">[[ ${request.uuid} ]]</td>
<td th:if="${request.report}">
<a th:href="@{/report/{id}(id=${request.uuid})}">[[ ${request.uuid} ]]</a>
<th:block sec:authorize="not authenticated">[[ ${request.uuid} ]]</th:block>
<a th:href="@{/report/{id}(id=${request.uuid})}" sec:authorize="authenticated">[[ ${request.uuid} ]]</a>
</td>
<td><time th:datetime="${request.processedAt}">[[ ${request.processedAt} ]]</time></td>
<td class="patient-id" sec:authorize="authenticated">[[ ${request.patientId} ]]</td>
<td class="patient-id" th:if="${patientId != null}" sec:authorize="authenticated">
[[ ${request.patientId} ]]
</td>
<td class="patient-id" th:if="${patientId == null}" sec:authorize="authenticated">
<a th:href="@{/patient/{pid}(pid=${request.patientId})}">[[ ${request.patientId} ]]</a>
</td>
<td class="patient-id" sec:authorize="not authenticated">***</td>
</tr>
</tbody>
@ -68,6 +88,12 @@
}
};
});
const eventSource = new EventSource('statistics/events');
eventSource.addEventListener('newrequest', event => {
console.log(event);
document.getElementById('reload-notify').style.display = 'inline-flex';
});
</script>
</body>
</html>

View File

@ -13,9 +13,11 @@
<div class="centered notification error" th:if="${param.error}">Anmeldung nicht erfolgreich</div>
<div class="centered notification success" th:if="${param.logout}">Sie haben sich abgemeldet</div>
<form method="post" th:action="@{/login}">
<input type="text" id="username" name="username" class="form-control" placeholder="Username" required="" autofocus="">
<input type="password" id="password" name="password" class="form-control" placeholder="Password" required="">
<button class="" type="submit">Anmelden</button>
<input type="text" id="username" name="username" class="form-control" placeholder="Username" required="" autofocus="" />
<input type="password" id="password" name="password" class="form-control" placeholder="Password" required="" />
<button type="submit">Anmelden</button>
<hr th:if="${not oidcLogins.isEmpty()}" />
<a th:each="oidcLogin : ${oidcLogins}" class="btn" th:href="@{/oauth2/authorization/{provider}(provider=${oidcLogin.component1()})}">OIDC Login - [[ ${oidcLogin.component2()} ]]</a>
</form>
</div>
</main>

View File

@ -38,7 +38,7 @@
</section>
</main>
<script th:src="@{/echarts.min.js}"></script>
<script th:src="@{/webjars/echarts/dist/echarts.min.js}"></script>
<script th:src="@{/scripts.js}"></script>
<script>
window.onload = () => {

View File

@ -0,0 +1,154 @@
/*
* This file is part of ETL-Processor
*
* Copyright (c) 2024 Comprehensive Cancer Center Mainfranken, Datenintegrationszentrum Philipps-Universität Marburg and Contributors
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as published
* by the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Affero General Public License for more details.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <https://www.gnu.org/licenses/>.
*/
package dev.dnpm.etl.processor.services
import org.assertj.core.api.Assertions.assertThat
import org.junit.jupiter.api.BeforeEach
import org.junit.jupiter.api.Test
import org.junit.jupiter.api.extension.ExtendWith
import org.mockito.ArgumentCaptor
import org.mockito.ArgumentMatchers.anyLong
import org.mockito.ArgumentMatchers.anyString
import org.mockito.Mock
import org.mockito.junit.jupiter.MockitoExtension
import org.mockito.kotlin.*
import org.springframework.security.crypto.password.PasswordEncoder
import org.springframework.security.provisioning.InMemoryUserDetailsManager
import java.util.*
import java.util.function.Consumer
@ExtendWith(MockitoExtension::class)
class TokenServiceTest {
private lateinit var userDetailsManager: InMemoryUserDetailsManager
private lateinit var passwordEncoder: PasswordEncoder
private lateinit var tokenRepository: TokenRepository
private lateinit var tokenService: TokenService
@BeforeEach
fun setup(
@Mock userDetailsManager: InMemoryUserDetailsManager,
@Mock passwordEncoder: PasswordEncoder,
@Mock tokenRepository: TokenRepository
) {
this.userDetailsManager = userDetailsManager
this.passwordEncoder = passwordEncoder
this.tokenRepository = tokenRepository
this.tokenService = TokenService(userDetailsManager, passwordEncoder, tokenRepository)
}
@Test
fun shouldEncodePasswordForNewToken() {
doAnswer { "{test}verysecret" }.whenever(passwordEncoder).encode(anyString())
val actual = this.tokenService.addToken("Test Token")
assertThat(actual).satisfies(
Consumer { assertThat(it.isSuccess).isTrue() },
Consumer { assertThat(it.getOrNull()).matches("testtoken:[A-Za-z0-9]{48}$") }
)
}
@Test
fun shouldContainAlphanumTokenUserPart() {
doAnswer { "{test}verysecret" }.whenever(passwordEncoder).encode(anyString())
val actual = this.tokenService.addToken("Test Token")
assertThat(actual).satisfies(
Consumer { assertThat(it.isSuccess).isTrue() },
Consumer { assertThat(it.getOrDefault("")).startsWith("testtoken:") }
)
}
@Test
fun shouldNotAllowSameTokenUserPartTwice() {
doReturn(true).whenever(userDetailsManager).userExists(anyString())
val actual = this.tokenService.addToken("Test Token")
assertThat(actual).satisfies(Consumer { assertThat(it.isFailure).isTrue() })
verify(tokenRepository, never()).save(any())
}
@Test
fun shouldSaveNewToken() {
doAnswer { "{test}verysecret" }.whenever(passwordEncoder).encode(anyString())
val actual = this.tokenService.addToken("Test Token")
val captor = ArgumentCaptor.forClass(Token::class.java)
verify(tokenRepository, times(1)).save(captor.capture())
assertThat(actual).satisfies(Consumer { assertThat(it.isSuccess).isTrue() })
assertThat(captor.value).satisfies(
Consumer { assertThat(it.name).isEqualTo("Test Token") },
Consumer { assertThat(it.username).isEqualTo("testtoken") },
Consumer { assertThat(it.password).isEqualTo("{test}verysecret") }
)
}
@Test
fun shouldDeleteExistingToken() {
doAnswer {
val id = it.arguments[0] as Long
Optional.of(Token(id, "Test Token", "testtoken", "{test}hsdajfgadskjhfgsdkfjg"))
}.whenever(tokenRepository).findById(anyLong())
this.tokenService.deleteToken(42)
val stringCaptor = ArgumentCaptor.forClass(String::class.java)
verify(userDetailsManager, times(1)).deleteUser(stringCaptor.capture())
assertThat(stringCaptor.value).isEqualTo("testtoken")
val tokenCaptor = ArgumentCaptor.forClass(Token::class.java)
verify(tokenRepository, times(1)).delete(tokenCaptor.capture())
assertThat(tokenCaptor.value.id).isEqualTo(42)
}
@Test
fun shouldReturnAllTokensFromRepository() {
val expected = listOf(
Token(1, "Test Token 1", "testtoken1", "{test}hsdajfgadskjhfgsdkfjg"),
Token(2, "Test Token 2", "testtoken2", "{test}asdasdasdasdasdasdasd")
)
doReturn(expected).whenever(tokenRepository).findAll()
assertThat(tokenService.findAll()).isEqualTo(expected)
}
@Test
fun shouldAddAllTokensFromRepositoryToUserDataManager() {
val expected = listOf(
Token(1, "Test Token 1", "testtoken1", "{test}hsdajfgadskjhfgsdkfjg"),
Token(2, "Test Token 2", "testtoken2", "{test}asdasdasdasdasdasdasd")
)
doReturn(expected).whenever(tokenRepository).findAll()
tokenService.setup()
verify(userDetailsManager, times(expected.size)).createUser(any())
}
}