1
0
mirror of https://github.com/pcvolkmer/etl-processor.git synced 2025-07-17 12:52:54 +00:00

31 Commits

Author SHA1 Message Date
f5aff88758 fix: throw unexpected exception in case gaps returns other than 2xx Code 2025-07-17 11:29:55 +02:00
d34127701c chore: prepare test submission for genomDe 2025-07-17 11:13:36 +02:00
0475bed377 fix: add missing return statement 2025-07-17 09:36:08 +02:00
58e46b4736 fix: improved error response in case gPas is not available. 2025-07-17 09:14:06 +02:00
dc6a4c6cb9 refactor: moved consent evaluation to new ConsentProcessor.kt 2025-07-16 15:20:26 +02:00
38eec60189 refactor: moved consent processing from RequestProcessor.kt to new class ConsentProcessor.kt 2025-07-16 11:02:11 +02:00
600ac84ff3 fix: serialize fhir consent resources to string, object map 2025-07-15 17:05:26 +02:00
be592b1a2a refactor: simplify consent check 2025-07-11 14:57:23 +02:00
90d1378e12 fix: pseudonymize patient reference at embedded consent resources 2025-07-10 15:15:14 +02:00
cc525d3f4f fix: added missing genomeDe consent version and date; moved embedding consent resources into base class of GicsConsentService 2025-07-10 11:43:37 +02:00
48fd491278 fix: ICheckConsent get genomDE consent 2025-07-10 09:48:10 +02:00
3a2f37c34c fix: serialization issue with fhir consent and embedding research consent 2025-07-09 19:08:42 +02:00
357abe2b05 fix: endpoint gPas connection check 2025-07-09 18:00:06 +02:00
0e48207e20 test: update fake patient test data 2025-07-09 15:09:34 +02:00
d343b79caa fix: reimplement broad consent and genomDE consent check and embedding them into mtb data 2025-07-09 13:26:29 +02:00
f855df5e5f Merge branch 'refs/heads/master' into 63-check-consent-status 2025-06-26 14:37:00 +02:00
27f81ab30e fix: check consent unknownStateIsConsideredAsDecline = false 2025-06-26 14:16:14 +02:00
9ad4466d69 feat: Broad Consent and GenomDE Consent can be embedded into mtb file 2025-06-25 16:16:35 +02:00
92f34459c3 chore: add HL7 Fhir serialization to ObjectMapper, so we can add FHIR consent resources to mtb file 2025-06-25 14:17:47 +02:00
60a1a351b1 refactor: rename method getIsConsentedParam -> getIsConsentedRequestParam 2025-06-24 12:45:22 +02:00
3686fd84a8 test: added test data -> gICS responses permit and deny 2025-06-24 12:03:28 +02:00
10cbf49777 feat: GicsConsentService can request consent resources from gICS based on domain (WIP) 2025-06-23 16:51:23 +02:00
1eefd1b27a Merge branch 'refs/heads/master' into 63-check-consent-status 2025-06-23 08:55:11 +02:00
d991f2a94d fix: added missing gIcs connection status to configuration view * also small refactoring 2025-05-14 09:22:59 +02:00
77df6f38ec refactor: renamed misleading enum value TtpConsentStatus.IGNORED to TtpConsentStatus.UNKNOWN_CHECK_FILE also impl. ConsentCheckedIgnored has been renamed to ConsentCheckFileBased - since consent status should never be ignored.
Also renamed ICheckConsent.isConsented ICheckConsent.getTtpConsentStatus since new method name is more descriptive.
2025-05-12 11:25:45 +02:00
542dc61811 fix: * Status prüfung gIcs überarbeitet.* Tests für GicsConsentService implementiert. * Überschreiben des MTB-File Consent, falls gICS aktiviert ist 2025-05-05 12:28:54 +02:00
2a28a4b3d4 doc: Konfiguration gICS Prüfung Einwilligungsstatus beschrieben 2025-05-05 12:25:44 +02:00
41d0a38c1d chore: some refactoring merge ttp consent status missing and rejected into one CONSENT_MISSING_OR_REJECTED 2025-04-28 15:10:20 +02:00
8517ba749c Merge branch 'master' into check-consent-status
# Conflicts:
#	build.gradle.kts
#	src/main/kotlin/dev/dnpm/etl/processor/config/AppConfiguration.kt
#	src/main/kotlin/dev/dnpm/etl/processor/input/MtbFileRestController.kt
#	src/main/kotlin/dev/dnpm/etl/processor/services/RequestProcessor.kt
#	src/test/kotlin/dev/dnpm/etl/processor/input/KafkaInputListenerTest.kt
#	src/test/kotlin/dev/dnpm/etl/processor/input/MtbFileRestControllerTest.kt
2025-04-28 12:33:39 +02:00
9074212575 chore: update .gitignore 2025-04-28 11:46:38 +02:00
b27670535f feat: check consent via gICS before process mtb file 2025-04-28 11:44:26 +02:00
44 changed files with 7919 additions and 1620 deletions

2
.gitignore vendored
View File

@ -39,3 +39,5 @@ out/
.vscode/
/dev/gpas*
/deploy/.env
/dev/gICS*
/dev/gPAS*

View File

@ -26,10 +26,12 @@ Konfigurationsparameter
### Modelvorhaben genomDE §64e
Zusätzlich zur Patienten Identifier Pseudonymisierung müssen Vorgangsummern generiert werden, die
jede Übertragung eindeutig identifizieren aber gleichzeitig dem Patienten zugeordnet werden können.
Dies lässt sich durch weitere Pseudonyme abbilden, allerdings werden pro Originalwert mehrere Pseudonyme benötigt.
Zu diesem Zweck muss in gPas eine **Multi-Pseudonym-Domäne** konfiguriert werden (siehe auch *APP_PSEUDONYMIZE_GPAS_CCDN*).
Um die voll Betriebsbereitschaft herzustellen, muss eine erfolgreiche Übertragung mit dem
Submission-Typ *Test* erfolgt sein. Über die Umgebungsvariable wird dieser Übertragungsmodus
aktiviert. Alle Datensätze mit erteilter Teilnahme am Modelvorhaben werden mit der Test-Kennung
übertragen.
`APP_GENOM_DE_TEST_SUBMISSION` -> `true` | `false` (falls fehlt wird `true` angenommen)
### Datenübermittlung über HTTP/REST
@ -107,10 +109,61 @@ Wurde die Verwendung von gPAS konfiguriert, so sind weitere Angaben zu konfiguri
* `APP_PSEUDONYMIZE_GPAS_URI`: URI der gPAS-Instanz inklusive Endpoint (z.B.
`http://localhost:8080/ttp-fhir/fhir/gpas/$$pseudonymizeAllowCreate`)
* `APP_PSEUDONYMIZE_GPAS_TARGET`: gPas Domänenname für Patienten ID
* `APP_PSEUDONYMIZE_GPAS_TARGET`: gPas Domänenname
* `APP_PSEUDONYMIZE_GPAS_USERNAME`: gPas Basic-Auth Benutzername
* `APP_PSEUDONYMIZE_GPAS_PASSWORD`: gPas Basic-Auth Passwort
* `APP_PSEUDONYMIZE_GPAS_CCDN`: gPas Multi-Pseudonym-Domäne für genomDE Vorgangsnummern (Clinical data node)
### Einwilligung gICS
Ab gIcs Version 2.13.0 kann
per [REST-Schnittstelle](https://simplifier.net/guide/ttp-fhir-gateway-ig/ImplementationGuide-markdown-Einwilligungsmanagement-Operations-isConsented?version=current)
der Einwilligungsstatus abgefragt werden.
Vor der MTB-Übertragung kann der zum Sendezeitpunkt verfügbarer Einwilligungsstatus über Endpunkt
*isConsented* abgefragt werden.
Falls Anbindung an gICS aktiviert wurde, wird der Einwilligungsstatus der MTB Datei ignoriert.
Stattdessen werden vorhandene Einwilligungen abgefragt und in die MTB Datei eingebettet.
Es werden zwei Einwilligungsdomänen unterstützt, eine für Broad Consent und als zweites GenomDE
Modelvorhaben §64e.
#### Hinweise
1. Die aktuelle Impl. nimmt an, dass die hinterlegten Domänen der Einwilligungen ausschließlich für
die genannten Art von Einwilligungen genutzt werden. Es finde keine weitere Filterung statt. Wir
fragen pro Domäne die Schnittstelle `CurrentPolicyStatesForPerson` - siehe
auch [IG TTP-FHIR Gateway
](https://www.ths-greifswald.de/wp-content/uploads/tools/fhirgw/ig/2024-3-0/ImplementationGuide-markdown-Einwilligungsmanagement-Operations-currentPolicyStatesForPerson.html)
ab.
2. Die Einwilligung wird für den Patienten-Identifier der MTB abgerufen und anschließend durch das
DNPM Pseudonym ersetzt.
3. Abfragen von Einwilligungen über gesonderte Pseudonyme anstatt des MTB-Identifiers fehlt in der
ersten Implementierung.
4. Bei Verarbeitung von MTB Version 1.x Inhalten ist eine positive Einwilligung für die
Weiterverarbeitung notwendig. Das Fehlen einer Einwilligung löst die Löschung des Patienten im
Brückenkopf aus.
#### Konfiguration
* `APP_CONSENT_GICS_ENABLED`: Aktiviert oder deaktiviert `true` oder `false`, `false` wenn nicht
gesetzt.
* `APP_CONSENT_GICS_CHECKGNOMEDE`: Aktiviert oder deaktiviert `true` oder `false`, `false` wenn
nicht gesetzt. Versuche Einwilligungsdaten zu GENOM DE Modelvorhaben über gIcs abzurufen.
* `APP_CONSENT_GICS_CHECKBROADCONSENT`: Aktiviert oder deaktiviert `true` oder `false`, `false` wenn
nicht gesetzt. Versuche Einwilligungsdaten zu Broad Consent über gIcs abzurufen.
* `APP_CONSENT_GICS_URI`: URI der gICS-Instanz (z.B. `http://localhost:8090/ttp-fhir/fhir/gics`)
* `APP_CONSENT_GICS_USERNAME`: gIcs Basic-Auth Benutzername
* `APP_CONSENT_GICS_PASSWORD`: gIcs Basic-Auth Passwort
* `APP_CONSENT_GICS_PERSONIDENTIFIERSYSTEM`: Derzeit wird nur die PID unterstützt. wenn leer wird
`https://ths-greifswald.de/fhir/gics/identifiers/Patienten-ID` angenommen
* `APP_CONSENT_GICS_BROADCONSENTDOMAINNAME`: Domäne in der gIcs Broad Consent Einwilligungen
verwaltet. Falls Wert leer, wird `MII` angenommen.
* `APP_CONSENT_GICS_GNOMDECONSENTDOMAINNAME`: Domäne in der gIcs GenomDE Modelvorhaben §64e
Einwilligungen verwaltet. Falls Wert leer, wird `GenomDE_MV` angenommen.
* `APP_CONSENT_GICS_POLICYCODE`: Die entscheidende Objekt-ID der zu prüfenden Einwilligung-Regel.
Falls leer wird `2.16.840.1.113883.3.1937.777.24.5.3.6` angenommen.
* `APP_CONSENT_GICS_POLICYSYSTEM`: Das System der Einwilligung-Regel der Objekt-IDs. Falls leer wird
`urn:oid:2.16.840.1.113883.3.1937.777.24.5.3` angenommen.
### Anmeldung mit einem Passwort
@ -488,4 +541,4 @@ Beim Ausführen der Integrationstests wird eine Testdatenbank in einem Docker-Co
Siehe hier auch die Klasse `AbstractTestcontainerTest` unter `src/integrationTest`.
Ein einfaches Entwickler-Setup inklusive DNPM:DIP ist mit Hilfe
von https://github.com/pcvolkmer/dnpmdip-devenv realisierbar.
von https://github.com/pcvolkmer/dnpmdip-devenv realisierbar.

View File

@ -16,6 +16,11 @@ services:
KAFKA_CFG_AUTO_CREATE_TOPICS_ENABLE: true
KAFKA_CFG_CONTROLLER_QUORUM_VOTERS: 0@kafka:9093
KAFKA_CFG_CONTROLLER_LISTENER_NAMES: CONTROLLER
healthcheck:
test: kafka-topics --bootstrap-server kafka:9092 --list
interval: 30s
timeout: 10s
retries: 3
## Use AKHQ as Kafka web frontend
akhq:
@ -53,4 +58,4 @@ services:
# environment:
# POSTGRES_DB: dev
# POSTGRES_USER: dev
# POSTGRES_PASSWORD: dev
# POSTGRES_PASSWORD: dev

View File

@ -2,31 +2,55 @@ version: '3.7'
services:
zoo1:
image: zookeeper:3.8.0
hostname: zoo1
zoo:
image: zookeeper:3.9.2
restart: unless-stopped
ports:
- "2181:2181"
environment:
ZOO_MY_ID: 1
ZOO_PORT: 2181
ZOO_SERVERS: server.1=zoo1:2888:3888;2181
ZOO_SERVERS: server.1=zoo:2888:3888;2181
kafka1:
image: confluentinc/cp-kafka:7.2.1
hostname: kafka1
kafka:
image: confluentinc/cp-kafka:7.6.1
ports:
- "9092:9092"
environment:
KAFKA_ADVERTISED_LISTENERS: LISTENER_DOCKER_INTERNAL://kafka1:19092,LISTENER_DOCKER_EXTERNAL://${DOCKER_HOST_IP:-127.0.0.1}:9092
KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: LISTENER_DOCKER_INTERNAL:PLAINTEXT,LISTENER_DOCKER_EXTERNAL:PLAINTEXT
KAFKA_ADVERTISED_LISTENERS: LISTENER_DOCKER_INTERNAL://kafka:19092,LISTENER_DOCKER_EXTERNAL://172.17.0.1:9093,LISTENER_EXTERNAL://127.0.0.1:9092
KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: LISTENER_DOCKER_INTERNAL:PLAINTEXT,LISTENER_DOCKER_EXTERNAL:PLAINTEXT,LISTENER_EXTERNAL:PLAINTEXT
KAFKA_INTER_BROKER_LISTENER_NAME: LISTENER_DOCKER_INTERNAL
KAFKA_ZOOKEEPER_CONNECT: "zoo1:2181"
KAFKA_ZOOKEEPER_CONNECT: zoo:2181
KAFKA_BROKER_ID: 1
KAFKA_LOG4J_LOGGERS: "kafka.controller=INFO,kafka.producer.async.DefaultEventHandler=INFO,state.change.logger=INFO"
KAFKA_LOG4J_LOGGERS: kafka.controller=INFO,kafka.producer.async.DefaultEventHandler=INFO,state.change.logger=INFO
KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
KAFKA_MESSAGE_MAX_BYTES: 5242880
KAFKA_REPLICA_FETCH_MAX_BYTES: 5242880
KAFKA_COMPRESSION_TYPE: gzip
depends_on:
- zoo1
- zoo
healthcheck:
test: kafka-topics --bootstrap-server kafka:9092 --list
interval: 30s
timeout: 10s
retries: 3
akhq:
image: tchiotludo/akhq:0.25.0
environment:
AKHQ_CONFIGURATION: |
akhq:
ui-options:
topic.show-all-consumer-groups: true
topic-data.sort: NEWEST
connections:
docker-kafka-server:
properties:
bootstrap.servers: "kafka:19092"
ports:
- "9000:8080"
depends_on:
- kafka
kafka-rest-proxy:
image: confluentinc/cp-kafka-rest:7.2.1
@ -40,8 +64,8 @@ services:
KAFKA_REST_HOST_NAME: kafka-rest-proxy
KAFKA_REST_BOOTSTRAP_SERVERS: PLAINTEXT://kafka1:19092
depends_on:
- zoo1
- kafka1
- zoo
- kafka
kafka-connect:
image: confluentinc/cp-kafka-connect:7.2.1
@ -67,24 +91,6 @@ services:
#volumes:
# - ./connectors:/etc/kafka-connect/jars/
depends_on:
- zoo1
- kafka1
- zoo
- kafka
- kafka-rest-proxy
akhq:
image: tchiotludo/akhq:0.21.0
environment:
AKHQ_CONFIGURATION: |
akhq:
connections:
docker-kafka-server:
properties:
bootstrap.servers: "kafka1:19092"
connect:
- name: "kafka-connect"
url: "http://kafka-connect:8083"
ports:
- "8084:8080"
depends_on:
- kafka1
- kafka-connect

View File

@ -23,6 +23,9 @@ import com.fasterxml.jackson.databind.ObjectMapper
import de.ukw.ccc.bwhc.dto.*
import dev.dnpm.etl.processor.anyValueClass
import dev.dnpm.etl.processor.config.AppSecurityConfiguration
import dev.dnpm.etl.processor.consent.ConsentByMtbFile
import dev.dnpm.etl.processor.consent.TtpConsentStatus
import dev.dnpm.etl.processor.consent.IGetConsent
import dev.dnpm.etl.processor.security.TokenRepository
import dev.dnpm.etl.processor.security.UserRoleRepository
import dev.dnpm.etl.processor.services.RequestProcessor
@ -31,10 +34,7 @@ import org.junit.jupiter.api.Nested
import org.junit.jupiter.api.Test
import org.junit.jupiter.api.extension.ExtendWith
import org.mockito.junit.jupiter.MockitoExtension
import org.mockito.kotlin.any
import org.mockito.kotlin.never
import org.mockito.kotlin.times
import org.mockito.kotlin.verify
import org.mockito.kotlin.*
import org.springframework.beans.factory.annotation.Autowired
import org.springframework.boot.test.autoconfigure.web.servlet.WebMvcTest
import org.springframework.http.MediaType
@ -54,7 +54,8 @@ import org.springframework.test.web.servlet.post
@ContextConfiguration(
classes = [
MtbFileRestController::class,
AppSecurityConfiguration::class
AppSecurityConfiguration::class,
ConsentByMtbFile::class, IGetConsent::class
]
)
@MockitoBean(types = [TokenRepository::class, RequestProcessor::class])
@ -63,7 +64,8 @@ import org.springframework.test.web.servlet.post
"app.pseudonymize.generator=BUILDIN",
"app.security.admin-user=admin",
"app.security.admin-password={noop}very-secret",
"app.security.enable-tokens=true"
"app.security.enable-tokens=true",
"app.consent.gics.enabled=false"
]
)
class MtbFileRestControllerTest {
@ -141,7 +143,7 @@ class MtbFileRestControllerTest {
status { isAccepted() }
}
verify(requestProcessor, times(1)).processDeletion(anyValueClass())
verify(requestProcessor, times(1)).processDeletion(anyValueClass(), eq(TtpConsentStatus.UNKNOWN_CHECK_FILE))
}
@Test
@ -152,7 +154,7 @@ class MtbFileRestControllerTest {
status { isUnauthorized() }
}
verify(requestProcessor, never()).processDeletion(anyValueClass())
verify(requestProcessor, never()).processDeletion(anyValueClass(), any())
}
@Nested
@ -163,7 +165,8 @@ class MtbFileRestControllerTest {
"app.security.admin-user=admin",
"app.security.admin-password={noop}very-secret",
"app.security.enable-tokens=true",
"app.security.enable-oidc=true"
"app.security.enable-oidc=true",
"app.consent.gics.enabled=false"
]
)
inner class WithOidcEnabled {

View File

@ -19,6 +19,7 @@
package dev.dnpm.etl.processor.pseudonym
import dev.dnpm.etl.processor.config.AppFhirConfig
import dev.dnpm.etl.processor.config.GPasConfigProperties
import org.assertj.core.api.Assertions.assertThat
import org.junit.jupiter.api.BeforeEach
@ -42,6 +43,7 @@ class GpasPseudonymGeneratorTest {
private lateinit var mockRestServiceServer: MockRestServiceServer
private lateinit var generator: GpasPseudonymGenerator
private lateinit var restTemplate: RestTemplate
private var appFhirConfig: AppFhirConfig = AppFhirConfig()
@BeforeEach
fun setup() {
@ -55,7 +57,8 @@ class GpasPseudonymGeneratorTest {
this.restTemplate = RestTemplate()
this.mockRestServiceServer = MockRestServiceServer.createServer(restTemplate)
this.generator = GpasPseudonymGenerator(gPasConfigProperties, retryTemplate, restTemplate)
this.generator =
GpasPseudonymGenerator(gPasConfigProperties, retryTemplate, restTemplate, appFhirConfig)
}
@Test
@ -64,7 +67,13 @@ class GpasPseudonymGeneratorTest {
method(HttpMethod.POST)
requestTo("https://localhost/ttp-fhir/fhir/gpas/\$pseudonymizeAllowCreate")
}.andRespond {
withStatus(HttpStatus.OK).body(getDummyResponseBody("1234", "test", "test1234ABCDEF567890"))
withStatus(HttpStatus.OK).body(
getDummyResponseBody(
"1234",
"test",
"test1234ABCDEF567890"
)
)
.createResponse(it)
}
@ -90,7 +99,10 @@ class GpasPseudonymGeneratorTest {
requestTo("https://localhost/ttp-fhir/fhir/gpas/\$pseudonymizeAllowCreate")
}.andRespond {
withStatus(HttpStatus.FOUND)
.header(HttpHeaders.LOCATION, "https://localhost/ttp-fhir/fhir/gpas/\$pseudonymizeAllowCreate")
.header(
HttpHeaders.LOCATION,
"https://localhost/ttp-fhir/fhir/gpas/\$pseudonymizeAllowCreate"
)
.createResponse(it)
}

View File

@ -22,6 +22,7 @@ package dev.dnpm.etl.processor.web
import dev.dnpm.etl.processor.config.AppConfiguration
import dev.dnpm.etl.processor.config.AppSecurityConfiguration
import dev.dnpm.etl.processor.monitoring.ConnectionCheckResult
import dev.dnpm.etl.processor.monitoring.GIcsConnectionCheckService
import dev.dnpm.etl.processor.monitoring.GPasConnectionCheckService
import dev.dnpm.etl.processor.monitoring.RestConnectionCheckService
import dev.dnpm.etl.processor.output.MtbFileSender
@ -89,7 +90,8 @@ abstract class MockSink : Sinks.Many<Boolean>
RequestProcessor::class,
TransformationService::class,
GPasConnectionCheckService::class,
RestConnectionCheckService::class
RestConnectionCheckService::class,
GIcsConnectionCheckService::class
]
)
class ConfigControllerTest {
@ -182,7 +184,13 @@ class ConfigControllerTest {
@Test
fun testShouldNotSaveTokenWithExstingName() {
whenever(tokenService.addToken(anyString())).thenReturn(Result.failure(RuntimeException("Testfailure")))
whenever(tokenService.addToken(anyString())).thenReturn(
Result.failure(
RuntimeException(
"Testfailure"
)
)
)
mockMvc.post("/configs/tokens") {
with(user("admin").roles("ADMIN"))
@ -303,7 +311,10 @@ class ConfigControllerTest {
val idCaptor = argumentCaptor<Long>()
val roleCaptor = argumentCaptor<Role>()
verify(userRoleService, times(1)).updateUserRole(idCaptor.capture(), roleCaptor.capture())
verify(userRoleService, times(1)).updateUserRole(
idCaptor.capture(),
roleCaptor.capture()
)
assertThat(idCaptor.firstValue).isEqualTo(42)
assertThat(roleCaptor.firstValue).isEqualTo(Role.ADMIN)
@ -341,23 +352,26 @@ class ConfigControllerTest {
@BeforeEach
fun setup(
applicationContext: WebApplicationContext,
applicationContext: WebApplicationContext
) {
this.webClient = MockMvcWebTestClient
.bindToApplicationContext(applicationContext).build()
}
@Test
fun testShouldRequestSSE() {
val expectedEvent = ConnectionCheckResult.GPasConnectionCheckResult(true, Instant.now(), Instant.now())
fun testShouldRequestGPasSSE() {
val expectedEvent =
ConnectionCheckResult.GPasConnectionCheckResult(true, Instant.now(), Instant.now())
connectionCheckUpdateProducer.tryEmitNext(expectedEvent)
connectionCheckUpdateProducer.emitComplete { _, _ -> true }
val result = webClient.get().uri("http://localhost/configs/events").accept(TEXT_EVENT_STREAM).exchange()
.expectStatus().isOk()
.expectHeader().contentType(TEXT_EVENT_STREAM)
.returnResult(ConnectionCheckResult.GPasConnectionCheckResult::class.java)
val result =
webClient.get().uri("http://localhost/configs/events").accept(TEXT_EVENT_STREAM)
.exchange()
.expectStatus().isOk()
.expectHeader().contentType(TEXT_EVENT_STREAM)
.returnResult(ConnectionCheckResult.GPasConnectionCheckResult::class.java)
StepVerifier.create(result.responseBody)
.expectNext(expectedEvent)

View File

@ -0,0 +1,36 @@
package dev.dnpm.etl.processor.consent;
import java.util.Date;
import org.hl7.fhir.r4.model.Bundle;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
public class ConsentByMtbFile implements IGetConsent {
private static final Logger log = LoggerFactory.getLogger(ConsentByMtbFile.class);
public ConsentByMtbFile() {
log.info("ConsentCheckFileBased initialized...");
}
@Override
public TtpConsentStatus getTtpBroadConsentStatus(String personIdentifierValue) {
return TtpConsentStatus.UNKNOWN_CHECK_FILE;
}
@Override
public Bundle getBroadConsent(String personIdentifierValue, Date requestDate) {
return IGetConsent.super.getBroadConsent(personIdentifierValue, requestDate);
}
@Override
public Bundle getGenomDeConsent(String personIdentifierValue, Date requestDate) {
return IGetConsent.super.getGenomDeConsent(personIdentifierValue, requestDate);
}
@Override
public Bundle currentConsentForPersonAndTemplate(String personIdentifierValue,
ConsentDomain targetConsentDomain, Date requestDate) {
return new Bundle();
}
}

View File

@ -0,0 +1,13 @@
package dev.dnpm.etl.processor.consent;
public enum ConsentDomain {
/**
* MII Broad consent
*/
BroadConsent,
/**
* GenomDe Modelvohaben §64e
*/
Modelvorhaben64e
}

View File

@ -0,0 +1,266 @@
package dev.dnpm.etl.processor.consent;
import ca.uhn.fhir.context.FhirContext;
import ca.uhn.fhir.parser.DataFormatException;
import dev.dnpm.etl.processor.config.AppFhirConfig;
import dev.dnpm.etl.processor.config.GIcsConfigProperties;
import java.util.Date;
import org.apache.commons.lang3.StringUtils;
import org.hl7.fhir.r4.model.BooleanType;
import org.hl7.fhir.r4.model.Bundle;
import org.hl7.fhir.r4.model.Coding;
import org.hl7.fhir.r4.model.DateType;
import org.hl7.fhir.r4.model.Identifier;
import org.hl7.fhir.r4.model.OperationOutcome;
import org.hl7.fhir.r4.model.Parameters;
import org.hl7.fhir.r4.model.Parameters.ParametersParameterComponent;
import org.hl7.fhir.r4.model.StringType;
import org.jetbrains.annotations.NotNull;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.http.HttpEntity;
import org.springframework.http.HttpHeaders;
import org.springframework.http.HttpMethod;
import org.springframework.http.MediaType;
import org.springframework.http.ResponseEntity;
import org.springframework.retry.TerminatedRetryException;
import org.springframework.retry.support.RetryTemplate;
import org.springframework.web.client.RestClientException;
import org.springframework.web.client.RestTemplate;
import org.springframework.web.util.UriComponentsBuilder;
public class GicsConsentService implements IGetConsent {
private final Logger log = LoggerFactory.getLogger(GicsConsentService.class);
public static final String IS_CONSENTED_ENDPOINT = "/$isConsented";
public static final String IS_POLICY_STATES_FOR_PERSON_ENDPOINT = "/$currentPolicyStatesForPerson";
private final RetryTemplate retryTemplate;
private final RestTemplate restTemplate;
private final FhirContext fhirContext;
private final HttpHeaders httpHeader;
private final GIcsConfigProperties gIcsConfigProperties;
private String url;
public GicsConsentService(GIcsConfigProperties gIcsConfigProperties,
RetryTemplate retryTemplate, RestTemplate restTemplate, AppFhirConfig appFhirConfig) {
this.retryTemplate = retryTemplate;
this.restTemplate = restTemplate;
this.fhirContext = appFhirConfig.fhirContext();
httpHeader = buildHeader(gIcsConfigProperties.getUsername(),
gIcsConfigProperties.getPassword());
this.gIcsConfigProperties = gIcsConfigProperties;
log.info("GicsConsentService initialized...");
}
public String getGicsUri(String endpoint) {
if (url == null) {
final String gIcsBaseUri = gIcsConfigProperties.getUri();
if (StringUtils.isBlank(gIcsBaseUri)) {
throw new IllegalArgumentException(
"gICS base URL is empty - should call gICS with false configuration.");
}
url = UriComponentsBuilder.fromUriString(gIcsBaseUri).path(endpoint)
.toUriString();
}
return url;
}
@NotNull
private static HttpHeaders buildHeader(String gPasUserName, String gPasPassword) {
var headers = new HttpHeaders();
headers.setContentType(MediaType.APPLICATION_XML);
if (StringUtils.isBlank(gPasUserName) || StringUtils.isBlank(gPasPassword)) {
return headers;
}
headers.setBasicAuth(gPasUserName, gPasPassword);
return headers;
}
protected static Parameters getIsConsentedRequestParam(GIcsConfigProperties configProperties,
String personIdentifierValue) {
var result = new Parameters();
result.addParameter(new ParametersParameterComponent().setName("personIdentifier").setValue(
new Identifier().setValue(personIdentifierValue)
.setSystem(configProperties.getPersonIdentifierSystem())));
result.addParameter(new ParametersParameterComponent().setName("domain")
.setValue(new StringType().setValue(configProperties.getBroadConsentDomainName())));
result.addParameter(new ParametersParameterComponent().setName("policy").setValue(
new Coding().setCode(configProperties.getBroadConsentPolicyCode())
.setSystem(configProperties.getBroadConsentPolicySystem())));
/*
* is mandatory parameter, but we ignore it via additional configuration parameter
* 'ignoreVersionNumber'.
*/
result.addParameter(new ParametersParameterComponent().setName("version")
.setValue(new StringType().setValue("1.1")));
/* add config parameter with:
* ignoreVersionNumber -> true ->> Reason is we cannot know which policy version each patient
* has possibly signed or not, therefore we are happy with any version found.
* unknownStateIsConsideredAsDecline -> true
*/
var config = new ParametersParameterComponent().setName("config").addPart(
new ParametersParameterComponent().setName("ignoreVersionNumber")
.setValue(new BooleanType().setValue(true))).addPart(
new ParametersParameterComponent().setName("unknownStateIsConsideredAsDecline")
.setValue(new BooleanType().setValue(false)));
result.addParameter(config);
return result;
}
protected String callGicsApi(Parameters parameter, String endpoint) {
var parameterAsXml = fhirContext.newXmlParser().encodeResourceToString(parameter);
HttpEntity<String> requestEntity = new HttpEntity<>(parameterAsXml, this.httpHeader);
ResponseEntity<String> responseEntity;
try {
var url = getGicsUri(endpoint);
responseEntity = retryTemplate.execute(
ctx -> restTemplate.exchange(url, HttpMethod.POST, requestEntity, String.class));
} catch (RestClientException e) {
var msg = String.format("Get consents status request failed reason: '%s",
e.getMessage());
log.error(msg);
return null;
} catch (TerminatedRetryException terminatedRetryException) {
var msg = String.format(
"Get consents status process has been terminated. termination reason: '%s",
terminatedRetryException.getMessage());
log.error(msg);
return null;
}
if (responseEntity.getStatusCode().is2xxSuccessful()) {
return responseEntity.getBody();
} else {
var msg = String.format(
"Trusted party system reached but request failed! code: '%s' response: '%s'",
responseEntity.getStatusCode(), responseEntity.getBody());
log.error(msg);
return null;
}
}
@Override
public TtpConsentStatus getTtpBroadConsentStatus(String personIdentifierValue) {
var parameter = GicsConsentService.getIsConsentedRequestParam(gIcsConfigProperties,
personIdentifierValue);
var consentStatusResponse = callGicsApi(parameter,
GicsConsentService.IS_CONSENTED_ENDPOINT);
return evaluateConsentResponse(consentStatusResponse);
}
public Bundle currentConsentForPersonAndTemplate(String personIdentifierValue,
ConsentDomain targetConsentDomain, Date requestDate) {
String consentDomain = getConsentDomain(targetConsentDomain);
var requestParameter = GicsConsentService.buildRequestParameterCurrentPolicyStatesForPerson(
gIcsConfigProperties, personIdentifierValue, requestDate, consentDomain);
var consentDataSerialized = callGicsApi(requestParameter,
GicsConsentService.IS_POLICY_STATES_FOR_PERSON_ENDPOINT);
if (consentDataSerialized == null) {
// error occurred - should not process further!
throw new IllegalStateException(
"consent data request failed - stopping processing! - try again or fix other problems first.");
}
var iBaseResource = fhirContext.newJsonParser()
.parseResource(consentDataSerialized);
if (iBaseResource instanceof OperationOutcome) {
// log error - very likely a configuration error
String errorMessage =
"Consent request failed! Check outcome:\n " + consentDataSerialized;
log.error(errorMessage);
throw new IllegalStateException(errorMessage);
} else if (iBaseResource instanceof Bundle) {
return (Bundle) iBaseResource;
} else {
String errorMessage = "Consent request failed! Unexpected response received! -> "
+ consentDataSerialized;
log.error(errorMessage);
throw new IllegalStateException(errorMessage);
}
}
@NotNull
private String getConsentDomain(ConsentDomain targetConsentDomain) {
String consentDomain;
switch (targetConsentDomain) {
case BroadConsent -> consentDomain = gIcsConfigProperties.getBroadConsentDomainName();
case Modelvorhaben64e ->
consentDomain = gIcsConfigProperties.getGenomDeConsentDomainName();
default -> throw new IllegalArgumentException(
"target ConsentDomain is missing but must be provided!");
}
return consentDomain;
}
protected static Parameters buildRequestParameterCurrentPolicyStatesForPerson(
GIcsConfigProperties gIcsConfigProperties, String personIdentifierValue, Date requestDate,
String targetDomain) {
var requestParameter = new Parameters();
requestParameter.addParameter(new ParametersParameterComponent().setName("personIdentifier")
.setValue(new Identifier().setValue(personIdentifierValue)
.setSystem(gIcsConfigProperties.getPersonIdentifierSystem())));
requestParameter.addParameter(new ParametersParameterComponent().setName("domain")
.setValue(new StringType().setValue(targetDomain)));
Parameters nestedConfigParameters = new Parameters();
nestedConfigParameters.addParameter(
new ParametersParameterComponent().setName("idMatchingType").setValue(
new Coding().setSystem(
"https://ths-greifswald.de/fhir/CodeSystem/gics/IdMatchingType")
.setCode("AT_LEAST_ONE"))).addParameter("ignoreVersionNumber", false)
.addParameter("unknownStateIsConsideredAsDecline", false)
.addParameter("requestDate", new DateType().setValue(requestDate));
requestParameter.addParameter(new ParametersParameterComponent().setName("config").addPart()
.setResource(nestedConfigParameters));
return requestParameter;
}
private TtpConsentStatus evaluateConsentResponse(String consentStatusResponse) {
if (consentStatusResponse == null) {
return TtpConsentStatus.FAILED_TO_ASK;
}
try {
var response = fhirContext.newJsonParser().parseResource(consentStatusResponse);
if (response instanceof Parameters responseParameters) {
var responseValue = responseParameters.getParameter("consented").getValue();
var isConsented = responseValue.castToBoolean(responseValue);
if (!isConsented.hasValue()) {
return TtpConsentStatus.FAILED_TO_ASK;
}
if (isConsented.booleanValue()) {
return TtpConsentStatus.BROAD_CONSENT_GIVEN;
} else {
return TtpConsentStatus.BROAD_CONSENT_MISSING_OR_REJECTED;
}
} else if (response instanceof OperationOutcome outcome) {
log.error("failed to get consent status from ttp. probably configuration error. "
+ "outcome: '{}'", fhirContext.newJsonParser().encodeToString(outcome));
}
} catch (DataFormatException dfe) {
log.error("failed to parse response to FHIR R4 resource.", dfe);
}
return TtpConsentStatus.FAILED_TO_ASK;
}
}

View File

@ -0,0 +1,52 @@
package dev.dnpm.etl.processor.consent;
import java.util.Date;
import org.hl7.fhir.r4.model.Bundle;
public interface IGetConsent {
/**
* Get broad consent status for a patient identifier
*
* @param personIdentifierValue patient identifier used for consent data
* @return status of broad consent
* @apiNote cannot not differ between not asked and rejected
*/
TtpConsentStatus getTtpBroadConsentStatus(String personIdentifierValue);
/**
* Get broad consent policies with respect to a request date
*
* @param personIdentifierValue patient identifier used for consent data
* @param requestDate target date until consent data should be considered
* @return consent policies as bundle; <p>if empty patient has not been asked, yet.</p>
*/
default Bundle getBroadConsent(String personIdentifierValue, Date requestDate) {
return currentConsentForPersonAndTemplate(personIdentifierValue, ConsentDomain.BroadConsent,
requestDate);
}
/**
* Get 'GenomDe Modelvorhaben §64e' consent policies with respect to a request date
*
* @param personIdentifierValue patient identifier used for consent data
* @param requestDate target date until consent data should be considered
* @return consent policies as bundle; <p>if empty patient has not been asked, yet.</p>
*/
default Bundle getGenomDeConsent(String personIdentifierValue, Date requestDate) {
return currentConsentForPersonAndTemplate(personIdentifierValue,
ConsentDomain.Modelvorhaben64e, requestDate);
}
/**
* Get consent policies with respect to a request date
*
* @param personIdentifierValue patient identifier used for consent data
* @param targetConsentDomain domain which should be used to request consent
* @param requestDate target date until consent data should be considered
* @return consent policies as bundle; <p>if empty patient has not been asked, yet.</p>
*/
Bundle currentConsentForPersonAndTemplate(String personIdentifierValue,
ConsentDomain targetConsentDomain, Date requestDate);
}

View File

@ -0,0 +1,38 @@
package dev.dnpm.etl.processor.consent;
public enum TtpConsentStatus {
/**
* Valid consent found
*/
BROAD_CONSENT_GIVEN,
/**
* Missing or rejected...actually unknown
*/
BROAD_CONSENT_MISSING_OR_REJECTED,
/**
* No Broad consent policy found
*/
BROAD_CONSENT_MISSING,
/**
* Research policy has been rejected
*/
BROAD_CONSENT_REJECTED,
GENOM_DE_CONSENT_SEQUENCING_PERMIT,
/**
* No GenomDE consent policy found
*/
GENOM_DE_CONSENT_MISSING,
/**
* GenomDE consent policy found, but has been rejected
*/
GENOM_DE_SEQUENCING_REJECTED,
/**
* Consent status is validate via file property 'consent.status'
*/
UNKNOWN_CHECK_FILE,
/**
* Due technical problems consent status is unknown
*/
FAILED_TO_ASK
}

View File

@ -21,6 +21,7 @@ package dev.dnpm.etl.processor.pseudonym;
import ca.uhn.fhir.context.FhirContext;
import ca.uhn.fhir.parser.IParser;
import dev.dnpm.etl.processor.config.AppFhirConfig;
import dev.dnpm.etl.processor.config.GPasConfigProperties;
import org.apache.commons.lang3.StringUtils;
import org.hl7.fhir.r4.model.Identifier;
@ -32,11 +33,14 @@ import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.http.*;
import org.springframework.retry.support.RetryTemplate;
import org.springframework.web.client.HttpClientErrorException.BadRequest;
import org.springframework.web.client.HttpClientErrorException.Unauthorized;
import org.springframework.web.client.RestClientException;
import org.springframework.web.client.RestTemplate;
public class GpasPseudonymGenerator implements Generator {
private final static FhirContext r4Context = FhirContext.forR4();
private final FhirContext r4Context;
private final String gPasUrl;
private final String psnTargetDomain;
private final HttpHeaders httpHeader;
@ -45,11 +49,13 @@ public class GpasPseudonymGenerator implements Generator {
private final RestTemplate restTemplate;
public GpasPseudonymGenerator(GPasConfigProperties gpasCfg, RetryTemplate retryTemplate, RestTemplate restTemplate) {
public GpasPseudonymGenerator(GPasConfigProperties gpasCfg, RetryTemplate retryTemplate,
RestTemplate restTemplate, AppFhirConfig appFhirConfig) {
this.retryTemplate = retryTemplate;
this.restTemplate = restTemplate;
this.gPasUrl = gpasCfg.getUri();
this.psnTargetDomain = gpasCfg.getTarget();
this.r4Context = appFhirConfig.fhirContext();
httpHeader = getHttpHeaders(gpasCfg.getUsername(), gpasCfg.getPassword());
log.debug(String.format("%s has been initialized", this.getClass().getName()));
@ -61,7 +67,7 @@ public class GpasPseudonymGenerator implements Generator {
var gPasRequestBody = getGpasRequestBody(id);
var responseEntity = getGpasPseudonym(gPasRequestBody);
var gPasPseudonymResult = (Parameters) r4Context.newJsonParser()
.parseResource(responseEntity.getBody());
.parseResource(responseEntity.getBody());
return unwrapPseudonym(gPasPseudonymResult);
}
@ -75,9 +81,9 @@ public class GpasPseudonymGenerator implements Generator {
}
final var identifier = (Identifier) parameters.get().getPart().stream()
.filter(a -> a.getName().equals("pseudonym"))
.findFirst()
.orElseGet(ParametersParameterComponent::new).getValue();
.filter(a -> a.getName().equals("pseudonym"))
.findFirst()
.orElseGet(ParametersParameterComponent::new).getValue();
// pseudonym
return sanitizeValue(identifier.getValue());
@ -97,38 +103,48 @@ public class GpasPseudonymGenerator implements Generator {
return psnValue.replaceAll(forbiddenCharsRegex, "_");
}
@NotNull
protected ResponseEntity<String> getGpasPseudonym(String gPasRequestBody) {
HttpEntity<String> requestEntity = new HttpEntity<>(gPasRequestBody, this.httpHeader);
ResponseEntity<String> responseEntity;
try {
responseEntity = retryTemplate.execute(
ctx -> restTemplate.exchange(gPasUrl, HttpMethod.POST, requestEntity,
String.class));
ResponseEntity<String> responseEntity = retryTemplate.execute(
ctx -> restTemplate.exchange(gPasUrl, HttpMethod.POST, requestEntity,
String.class));
if (responseEntity.getStatusCode().is2xxSuccessful()) {
log.debug("API request succeeded. Response: {}", responseEntity.getStatusCode());
} else {
log.warn("API request unsuccessful. Response: {}", requestEntity.getBody());
throw new PseudonymRequestFailed("API request unsuccessful gPas unsuccessful.");
return responseEntity;
}
} catch (RestClientException rce) {
if (rce instanceof BadRequest) {
String msg = "gPas or request configuration is incorrect. Please check both."
+ rce.getMessage();
log.debug(
msg);
throw new PseudonymRequestFailed(msg, rce);
}
if (rce instanceof Unauthorized) {
var msg = "gPas access credentials are invalid check your configuration. msg: '%s".formatted(
rce.getMessage());
log.error(msg);
throw new PseudonymRequestFailed(msg, rce);
}
return responseEntity;
} catch (Exception unexpected) {
throw new PseudonymRequestFailed(
"API request due unexpected error unsuccessful gPas unsuccessful.", unexpected);
"API request due unexpected error unsuccessful gPas unsuccessful.", unexpected);
}
throw new PseudonymRequestFailed(
"API request due unexpected error unsuccessful gPas unsuccessful.");
}
protected String getGpasRequestBody(String id) {
var requestParameters = new Parameters();
requestParameters.addParameter().setName("target")
.setValue(new StringType().setValue(psnTargetDomain));
.setValue(new StringType().setValue(psnTargetDomain));
requestParameters.addParameter().setName("original")
.setValue(new StringType().setValue(id));
.setValue(new StringType().setValue(id));
final IParser iParser = r4Context.newJsonParser();
return iParser.encodeResourceToString(requestParameters);
}

View File

@ -27,7 +27,8 @@ data class AppConfigProperties(
var bwhcUri: String?,
var transformations: List<TransformationProperties> = listOf(),
var maxRetryAttempts: Int = 3,
var duplicationDetection: Boolean = true
var duplicationDetection: Boolean = true,
var genomDeTestSubmission: Boolean = true
) {
companion object {
const val NAME = "app"
@ -56,6 +57,70 @@ data class GPasConfigProperties(
}
}
@ConfigurationProperties(GIcsConfigProperties.NAME)
data class GIcsConfigProperties(
/**
* Base URL to gICS System
*
*/
val uri: String?,
val username: String?,
val password: String?,
/**
* If value is 'true' valid consent at processing time is mandatory for transmission of DNPM
* files otherwise they will be flagged and skipped.
* If value 'false' or missing consent status is assumed to be valid.
*/
val enabled: Boolean?,
/**
* gICS specific system
* **/
val personIdentifierSystem: String =
"https://ths-greifswald.de/fhir/gics/identifiers/Patienten-ID",
/**
* Domain of broad consent resources
**/
val broadConsentDomainName: String = "MII",
/**
* Domain of Modelvorhaben 64e consent resources
**/
val genomDeConsentDomainName: String = "GenomDE_MV",
/**
* Value to expect in case of positiv consent
*/
val broadConsentPolicyCode: String = "2.16.840.1.113883.3.1937.777.24.5.3.6",
/**
* Consent Policy which should be used for consent check
*/
val broadConsentPolicySystem: String = "urn:oid:2.16.840.1.113883.3.1937.777.24.5.3",
/**
* Value to expect in case of positiv consent
*/
val genomeDePolicyCode: String = "sequencing",
/**
* Consent Policy which should be used for consent check
*/
val genomeDePolicySystem: String = "https://ths-greifswald.de/fhir/CodeSystem/gics/Policy/GenomDE_MV",
/**
* Consent version (fixed version)
*
*/
val genomeDeConsentVersion: String = "2.0"
) {
companion object {
const val NAME = "app.consent.gics"
}
}
@ConfigurationProperties(RestTargetProperties.NAME)
data class RestTargetProperties(
val uri: String?,

View File

@ -20,16 +20,17 @@
package dev.dnpm.etl.processor.config
import com.fasterxml.jackson.databind.ObjectMapper
import dev.dnpm.etl.processor.monitoring.ConnectionCheckResult
import dev.dnpm.etl.processor.monitoring.ConnectionCheckService
import dev.dnpm.etl.processor.monitoring.GPasConnectionCheckService
import dev.dnpm.etl.processor.monitoring.ReportService
import dev.dnpm.etl.processor.consent.ConsentByMtbFile
import dev.dnpm.etl.processor.consent.IGetConsent
import dev.dnpm.etl.processor.consent.GicsConsentService
import dev.dnpm.etl.processor.monitoring.*
import dev.dnpm.etl.processor.pseudonym.AnonymizingGenerator
import dev.dnpm.etl.processor.pseudonym.Generator
import dev.dnpm.etl.processor.pseudonym.GpasPseudonymGenerator
import dev.dnpm.etl.processor.pseudonym.PseudonymizeService
import dev.dnpm.etl.processor.security.TokenRepository
import dev.dnpm.etl.processor.security.TokenService
import dev.dnpm.etl.processor.services.ConsentProcessor
import dev.dnpm.etl.processor.services.Transformation
import dev.dnpm.etl.processor.services.TransformationService
import org.slf4j.LoggerFactory
@ -60,7 +61,8 @@ import kotlin.time.toJavaDuration
value = [
AppConfigProperties::class,
PseudonymizeConfigProperties::class,
GPasConfigProperties::class
GPasConfigProperties::class,
GIcsConfigProperties::class
]
)
@EnableScheduling
@ -73,13 +75,27 @@ class AppConfiguration {
return RestTemplate()
}
@ConditionalOnProperty(value = ["app.pseudonymize.generator"], havingValue = "GPAS")
@Bean
fun gpasPseudonymGenerator(configProperties: GPasConfigProperties, retryTemplate: RetryTemplate, restTemplate: RestTemplate): Generator {
return GpasPseudonymGenerator(configProperties, retryTemplate, restTemplate)
fun appFhirConfig(): AppFhirConfig {
return AppFhirConfig()
}
@ConditionalOnProperty(value = ["app.pseudonymize.generator"], havingValue = "BUILDIN", matchIfMissing = true)
@ConditionalOnProperty(value = ["app.pseudonymize.generator"], havingValue = "GPAS")
@Bean
fun gpasPseudonymGenerator(
configProperties: GPasConfigProperties,
retryTemplate: RetryTemplate,
restTemplate: RestTemplate,
appFhirConfig: AppFhirConfig
): Generator {
return GpasPseudonymGenerator(configProperties, retryTemplate, restTemplate, appFhirConfig)
}
@ConditionalOnProperty(
value = ["app.pseudonymize.generator"],
havingValue = "BUILDIN",
matchIfMissing = true
)
@Bean
fun buildinPseudonymGenerator(): Generator {
return AnonymizingGenerator()
@ -94,17 +110,21 @@ class AppConfiguration {
}
@Bean
fun reportService(objectMapper: ObjectMapper): ReportService {
return ReportService(objectMapper)
fun reportService(): ReportService {
return ReportService(getObjectMapper())
}
@Bean
fun getObjectMapper(): ObjectMapper {
return JacksonConfig().objectMapper()
}
@Bean
fun transformationService(
objectMapper: ObjectMapper,
configProperties: AppConfigProperties
): TransformationService {
logger.info("Apply ${configProperties.transformations.size} transformation rules")
return TransformationService(objectMapper, configProperties.transformations.map {
return TransformationService(getObjectMapper(), configProperties.transformations.map {
Transformation.of(it.path) from it.from to it.to
})
}
@ -123,7 +143,11 @@ class AppConfiguration {
callback: RetryCallback<T, E>,
throwable: Throwable
) {
logger.warn("Error occured: {}. Retrying {}", throwable.message, context.retryCount)
logger.warn(
"Error occured: {}. Retrying {}",
throwable.message,
context.retryCount
)
}
})
.build()
@ -131,7 +155,11 @@ class AppConfiguration {
@ConditionalOnProperty(value = ["app.security.enable-tokens"], havingValue = "true")
@Bean
fun tokenService(userDetailsManager: InMemoryUserDetailsManager, passwordEncoder: PasswordEncoder, tokenRepository: TokenRepository): TokenService {
fun tokenService(
userDetailsManager: InMemoryUserDetailsManager,
passwordEncoder: PasswordEncoder,
tokenRepository: TokenRepository
): TokenService {
return TokenService(userDetailsManager, passwordEncoder, tokenRepository)
}
@ -152,7 +180,11 @@ class AppConfiguration {
gPasConfigProperties: GPasConfigProperties,
connectionCheckUpdateProducer: Sinks.Many<ConnectionCheckResult>
): ConnectionCheckService {
return GPasConnectionCheckService(restTemplate, gPasConfigProperties, connectionCheckUpdateProducer)
return GPasConnectionCheckService(
restTemplate,
gPasConfigProperties,
connectionCheckUpdateProducer
)
}
@ConditionalOnProperty(value = ["app.pseudonymizer"], havingValue = "GPAS")
@ -163,12 +195,68 @@ class AppConfiguration {
gPasConfigProperties: GPasConfigProperties,
connectionCheckUpdateProducer: Sinks.Many<ConnectionCheckResult>
): ConnectionCheckService {
return GPasConnectionCheckService(restTemplate, gPasConfigProperties, connectionCheckUpdateProducer)
return GPasConnectionCheckService(
restTemplate,
gPasConfigProperties,
connectionCheckUpdateProducer
)
}
@Bean
fun jdbcConfiguration(): AbstractJdbcConfiguration {
return AppJdbcConfiguration()
}
@Bean
@ConditionalOnProperty(name = ["app.consent.gics.enabled"], havingValue = "true")
fun gicsConsentService(
gIcsConfigProperties: GIcsConfigProperties,
retryTemplate: RetryTemplate, restTemplate: RestTemplate, appFhirConfig: AppFhirConfig
): IGetConsent {
return GicsConsentService(
gIcsConfigProperties,
retryTemplate,
restTemplate,
appFhirConfig
)
}
@Bean
@ConditionalOnProperty(name = ["app.consent.gics.enabled"], havingValue = "true")
fun gicsConsentProcessor(
configProperties: AppConfigProperties,
gIcsConfigProperties: GIcsConfigProperties,
getObjectMapper: ObjectMapper,
appFhirConfig: AppFhirConfig,
gicsConsentService: IGetConsent
): ConsentProcessor {
return ConsentProcessor(
configProperties,
gIcsConfigProperties,
getObjectMapper,
appFhirConfig.fhirContext(),
gicsConsentService
)
}
@ConditionalOnProperty(name = ["app.consent.gics.enabled"], havingValue = "true")
@Bean
fun gIcsConnectionCheckService(
restTemplate: RestTemplate,
gIcsConfigProperties: GIcsConfigProperties,
connectionCheckUpdateProducer: Sinks.Many<ConnectionCheckResult>
): ConnectionCheckService {
return GIcsConnectionCheckService(
restTemplate,
gIcsConfigProperties,
connectionCheckUpdateProducer
)
}
@Bean
@ConditionalOnMissingBean
fun iGetConsentService(): IGetConsent {
return ConsentByMtbFile()
}
}

View File

@ -0,0 +1,16 @@
package dev.dnpm.etl.processor.config
import ca.uhn.fhir.context.FhirContext
import org.springframework.context.annotation.Bean
import org.springframework.context.annotation.Configuration
@Configuration
class AppFhirConfig {
private val fhirCtx: FhirContext = FhirContext.forR4()
@Bean
fun fhirContext(): FhirContext {
return fhirCtx
}
}

View File

@ -0,0 +1,18 @@
package dev.dnpm.etl.processor.config
import com.fasterxml.jackson.core.JsonParser
import com.fasterxml.jackson.databind.DeserializationContext
import com.fasterxml.jackson.databind.JsonDeserializer
import com.fasterxml.jackson.databind.JsonNode
import org.hl7.fhir.r4.model.Consent
class ConsentResourceDeserializer : JsonDeserializer<Consent>() {
override fun deserialize(p: JsonParser?, ctxt: DeserializationContext?): Consent {
val jsonNode = p?.readValueAsTree<JsonNode>()
val json = jsonNode?.toString()
return JacksonConfig.fhirContext().newJsonParser().parseResource(json) as Consent
}
}

View File

@ -0,0 +1,15 @@
package dev.dnpm.etl.processor.config
import com.fasterxml.jackson.core.JsonGenerator
import com.fasterxml.jackson.databind.JsonSerializer
import com.fasterxml.jackson.databind.SerializerProvider
import org.hl7.fhir.r4.model.Consent
class ConsentResourceSerializer : JsonSerializer<Consent>() {
override fun serialize(
value: Consent, gen: JsonGenerator, serializers: SerializerProvider
) {
val json = JacksonConfig.fhirContext().newJsonParser().encodeResourceToString(value)
gen.writeRawValue(json)
}
}

View File

@ -0,0 +1,12 @@
package dev.dnpm.etl.processor.config
import com.fasterxml.jackson.databind.module.SimpleModule
import org.hl7.fhir.r4.model.Consent
class FhirResourceModule : SimpleModule() {
init {
addSerializer(Consent::class.java, ConsentResourceSerializer())
addDeserializer(Consent::class.java, ConsentResourceDeserializer())
}
}

View File

@ -0,0 +1,27 @@
package dev.dnpm.etl.processor.config
import ca.uhn.fhir.context.FhirContext
import org.springframework.context.annotation.Bean
import org.springframework.context.annotation.Configuration
import com.fasterxml.jackson.databind.ObjectMapper
import com.fasterxml.jackson.databind.SerializationFeature
import com.fasterxml.jackson.datatype.jsr310.JavaTimeModule
@Configuration
class JacksonConfig {
companion object {
var fhirContext: FhirContext = FhirContext.forR4()
@JvmStatic
fun fhirContext(): FhirContext {
return fhirContext
}
}
@Bean
fun objectMapper(): ObjectMapper = ObjectMapper().registerModule(FhirResourceModule())
.disable(SerializationFeature.WRITE_DATES_AS_TIMESTAMPS).registerModule(
JavaTimeModule()
)
}

View File

@ -25,6 +25,7 @@ import de.ukw.ccc.bwhc.dto.MtbFile
import dev.dnpm.etl.processor.CustomMediaType
import dev.dnpm.etl.processor.PatientId
import dev.dnpm.etl.processor.RequestId
import dev.dnpm.etl.processor.consent.TtpConsentStatus
import dev.dnpm.etl.processor.services.RequestProcessor
import org.apache.kafka.clients.consumer.ConsumerRecord
import org.slf4j.LoggerFactory
@ -76,9 +77,13 @@ class KafkaInputListener(
} else {
logger.debug("Accepted MTB File and process deletion")
if (requestId.isBlank()) {
requestProcessor.processDeletion(patientId)
requestProcessor.processDeletion(patientId, TtpConsentStatus.UNKNOWN_CHECK_FILE)
} else {
requestProcessor.processDeletion(patientId, requestId)
requestProcessor.processDeletion(
patientId,
requestId,
TtpConsentStatus.UNKNOWN_CHECK_FILE
)
}
}
}

View File

@ -23,6 +23,8 @@ import de.ukw.ccc.bwhc.dto.Consent
import de.ukw.ccc.bwhc.dto.MtbFile
import dev.dnpm.etl.processor.CustomMediaType
import dev.dnpm.etl.processor.PatientId
import dev.dnpm.etl.processor.consent.IGetConsent
import dev.dnpm.etl.processor.consent.TtpConsentStatus
import dev.dnpm.etl.processor.services.RequestProcessor
import dev.pcvolkmer.mv64e.mtb.Mtb
import org.slf4j.LoggerFactory
@ -33,7 +35,7 @@ import org.springframework.web.bind.annotation.*
@RestController
@RequestMapping(path = ["mtbfile", "mtb"])
class MtbFileRestController(
private val requestProcessor: RequestProcessor,
private val requestProcessor: RequestProcessor, private val iGetConsent: IGetConsent
) {
private val logger = LoggerFactory.getLogger(MtbFileRestController::class.java)
@ -43,20 +45,39 @@ class MtbFileRestController(
return ResponseEntity.ok("Test")
}
@PostMapping( consumes = [ MediaType.APPLICATION_JSON_VALUE ] )
@PostMapping(consumes = [MediaType.APPLICATION_JSON_VALUE])
fun mtbFile(@RequestBody mtbFile: MtbFile): ResponseEntity<Unit> {
if (mtbFile.consent.status == Consent.Status.ACTIVE) {
val consentStatusBooleanPair = checkConsentStatus(mtbFile)
val ttpConsentStatus = consentStatusBooleanPair.first
val isConsentOK = consentStatusBooleanPair.second
if (isConsentOK) {
logger.debug("Accepted MTB File (bwHC V1) for processing")
requestProcessor.processMtbFile(mtbFile)
} else {
logger.debug("Accepted MTB File (bwHC V1) and process deletion")
val patientId = PatientId(mtbFile.patient.id)
requestProcessor.processDeletion(patientId)
requestProcessor.processDeletion(patientId, ttpConsentStatus)
}
return ResponseEntity.accepted().build()
}
@PostMapping( consumes = [ CustomMediaType.APPLICATION_VND_DNPM_V2_MTB_JSON_VALUE] )
private fun checkConsentStatus(mtbFile: MtbFile): Pair<TtpConsentStatus, Boolean> {
var ttpConsentStatus = iGetConsent.getTtpBroadConsentStatus(mtbFile.patient.id)
val isConsentOK =
(ttpConsentStatus.equals(TtpConsentStatus.UNKNOWN_CHECK_FILE) && mtbFile.consent.status == Consent.Status.ACTIVE) ||
ttpConsentStatus.equals(
TtpConsentStatus.BROAD_CONSENT_GIVEN
)
if (ttpConsentStatus.equals(TtpConsentStatus.UNKNOWN_CHECK_FILE) && mtbFile.consent.status == Consent.Status.REJECTED) {
// in case ttp check is disabled - we propagate rejected status anyway
ttpConsentStatus = TtpConsentStatus.BROAD_CONSENT_MISSING_OR_REJECTED
}
return Pair(ttpConsentStatus, isConsentOK)
}
@PostMapping(consumes = [CustomMediaType.APPLICATION_VND_DNPM_V2_MTB_JSON_VALUE])
fun mtbFile(@RequestBody mtbFile: Mtb): ResponseEntity<Unit> {
logger.debug("Accepted MTB File (DNPM V2) for processing")
requestProcessor.processMtbFile(mtbFile)
@ -66,7 +87,7 @@ class MtbFileRestController(
@DeleteMapping(path = ["{patientId}"])
fun deleteData(@PathVariable patientId: String): ResponseEntity<Unit> {
logger.debug("Accepted patient ID to process deletion")
requestProcessor.processDeletion(PatientId(patientId))
requestProcessor.processDeletion(PatientId(patientId), TtpConsentStatus.UNKNOWN_CHECK_FILE)
return ResponseEntity.accepted().build()
}

View File

@ -20,6 +20,7 @@
package dev.dnpm.etl.processor.monitoring
import dev.dnpm.etl.processor.config.GIcsConfigProperties
import dev.dnpm.etl.processor.config.GPasConfigProperties
import dev.dnpm.etl.processor.config.RestTargetProperties
import jakarta.annotation.PostConstruct
@ -68,6 +69,12 @@ sealed class ConnectionCheckResult {
override val timestamp: Instant,
override val lastChange: Instant
) : ConnectionCheckResult()
data class GIcsConnectionCheckResult(
override val available: Boolean,
override val timestamp: Instant,
override val lastChange: Instant
) : ConnectionCheckResult()
}
class KafkaConnectionCheckService(
@ -207,4 +214,57 @@ class GPasConnectionCheckService(
override fun connectionAvailable(): ConnectionCheckResult.GPasConnectionCheckResult {
return this.result
}
}
class GIcsConnectionCheckService(
private val restTemplate: RestTemplate,
private val gIcsConfigProperties: GIcsConfigProperties,
@Qualifier("connectionCheckUpdateProducer")
private val connectionCheckUpdateProducer: Sinks.Many<ConnectionCheckResult>
) : ConnectionCheckService {
private var result = ConnectionCheckResult.GIcsConnectionCheckResult(false, Instant.now(), Instant.now())
@PostConstruct
@Scheduled(cron = "0 * * * * *")
fun check() {
result = try {
val uri = UriComponentsBuilder.fromUriString(
gIcsConfigProperties.uri.toString()).path("/metadata").build().toUri()
val headers = HttpHeaders()
headers.contentType = MediaType.APPLICATION_JSON
if (!gIcsConfigProperties.username.isNullOrBlank() && !gIcsConfigProperties.password.isNullOrBlank()) {
headers.setBasicAuth(gIcsConfigProperties.username, gIcsConfigProperties.password)
}
val available = restTemplate.exchange(
uri,
HttpMethod.GET,
HttpEntity<Void>(headers),
Void::class.java
).statusCode == HttpStatus.OK
ConnectionCheckResult.GIcsConnectionCheckResult(
available,
Instant.now(),
if (result.available == available) { result.lastChange } else { Instant.now() }
)
} catch (_: Exception) {
ConnectionCheckResult.GIcsConnectionCheckResult(
false,
Instant.now(),
if (!result.available) { result.lastChange } else { Instant.now() }
)
}
connectionCheckUpdateProducer.emitNext(
result,
Sinks.EmitFailureHandler.FAIL_FAST
)
}
override fun connectionAvailable(): ConnectionCheckResult.GIcsConnectionCheckResult {
return this.result
}
}

View File

@ -24,5 +24,6 @@ enum class RequestStatus(val value: String) {
WARNING("warning"),
ERROR("error"),
UNKNOWN("unknown"),
DUPLICATION("duplication")
DUPLICATION("duplication"),
NO_CONSENT("no-consent")
}

View File

@ -21,7 +21,9 @@ package dev.dnpm.etl.processor.pseudonym
import de.ukw.ccc.bwhc.dto.MtbFile
import dev.dnpm.etl.processor.PatientId
import dev.pcvolkmer.mv64e.mtb.ModelProjectConsent
import dev.pcvolkmer.mv64e.mtb.Mtb
import dev.pcvolkmer.mv64e.mtb.MvhMetadata
import org.apache.commons.codec.digest.DigestUtils
/** Replaces patient ID with generated patient pseudonym
@ -289,6 +291,16 @@ infix fun Mtb.pseudonymizeWith(pseudonymizeService: PseudonymizeService) {
this.followUps?.forEach {
it.patient.id = patientPseudonym
}
this.metadata?.researchConsents?.forEach { it ->
val entry = it ?: return@forEach
if (entry.contains("patient")) {
// here we expect only a patient reference any other data like display
// need to be removed, since may contain unsecure data
entry.remove("patient")
entry["patient"] = mapOf("reference" to "Patient/$patientPseudonym")
}
}
}
/**
@ -317,3 +329,23 @@ infix fun Mtb.anonymizeContentWith(pseudonymizeService: PseudonymizeService) {
// TODO all other properties
}
fun Mtb.ensureMetaDataIsInitialized() {
// init metadata if necessary
if (this.metadata == null) {
val mvhMetadata = MvhMetadata.builder().build()
this.metadata = mvhMetadata
}
if (this.metadata.researchConsents == null) {
this.metadata.researchConsents = mutableListOf()
}
if (this.metadata.modelProjectConsent == null) {
this.metadata.modelProjectConsent = ModelProjectConsent()
this.metadata.modelProjectConsent.provisions = mutableListOf()
} else
if (this.metadata.modelProjectConsent.provisions != null) {
// make sure list can be changed
this.metadata.modelProjectConsent.provisions =
this.metadata.modelProjectConsent.provisions.toMutableList()
}
}

View File

@ -0,0 +1,276 @@
package dev.dnpm.etl.processor.services
import ca.uhn.fhir.context.FhirContext
import com.fasterxml.jackson.core.JsonProcessingException
import com.fasterxml.jackson.core.type.TypeReference
import com.fasterxml.jackson.databind.ObjectMapper
import dev.dnpm.etl.processor.config.AppConfigProperties
import dev.dnpm.etl.processor.config.GIcsConfigProperties
import dev.dnpm.etl.processor.consent.ConsentDomain
import dev.dnpm.etl.processor.consent.IGetConsent
import dev.dnpm.etl.processor.pseudonym.ensureMetaDataIsInitialized
import dev.pcvolkmer.mv64e.mtb.*
import org.apache.commons.lang3.NotImplementedException
import org.hl7.fhir.r4.model.*
import org.hl7.fhir.r4.model.Bundle.BundleEntryComponent
import org.hl7.fhir.r4.model.Coding
import org.hl7.fhir.r4.model.Consent.ConsentState
import org.hl7.fhir.r4.model.Consent.ProvisionComponent
import org.slf4j.Logger
import org.slf4j.LoggerFactory
import org.springframework.stereotype.Service
import java.io.IOException
import java.time.Clock
import java.time.Instant
import java.util.*
@Service
class ConsentProcessor(
private val appConfigProperties: AppConfigProperties,
private val gIcsConfigProperties: GIcsConfigProperties,
private val objectMapper: ObjectMapper,
private val fhirContext: FhirContext,
private val consentService: IGetConsent?
) {
private var logger: Logger = LoggerFactory.getLogger("ConsentProcessor")
/**
* In case an instance of {@link ICheckConsent} is active, consent will be embedded and checked.
*
* Logik:
* * <c>true</c> IF consent check is disabled.
* * <c>true</c> IF broad consent (BC) has been given.
* * <c>true</c> BC has been asked AND declined but genomDe consent has been consented.
* * ELSE <c>false</c> is returned.
*
* @param mtbFile File v2 (will be enriched with consent data)
* @return true if consent is given
*
*/
fun consentGatedCheckAndTryEmbedding(mtbFile: Mtb): Boolean {
if (consentService == null) {
// consent check seems to be disabled
return true
}
mtbFile.ensureMetaDataIsInitialized()
val personIdentifierValue = mtbFile.patient.id
val requestDate = Date.from(Instant.now(Clock.systemUTC()))
// 1. Broad consent Entry exists?
// 1.1. -> yes and research consent is given -> send mtb file
// 1.2. -> no -> return status error - consent has not been asked
// 2. -> Broad consent found but rejected -> is GenomDe consent provision 'sequencing' given?
// 2.1 -> yes -> send mtb file
// 2.2 -> no -> warn/info no consent given
/*
* broad consent
*/
val broadConsent = consentService.getBroadConsent(personIdentifierValue, requestDate)
val broadConsentHasBeenAsked = !broadConsent.entry.isEmpty()
// fast exit - if patient has not been asked, we can skip and exit
if (!broadConsentHasBeenAsked) return false
val genomeDeConsent = consentService.getGenomDeConsent(
personIdentifierValue, requestDate
)
addGenomeDbProvisions(mtbFile, genomeDeConsent)
if (!genomeDeConsent.entry.isEmpty()) setGenomDeSubmissionType(mtbFile)
embedBroadConsentResources(mtbFile, broadConsent)
val broadConsentStatus = getProvisionTypeByPolicyCode(
broadConsent, requestDate, ConsentDomain.BroadConsent
)
val genomDeSequencingStatus = getProvisionTypeByPolicyCode(
genomeDeConsent, requestDate, ConsentDomain.Modelvorhaben64e
)
if (Consent.ConsentProvisionType.NULL == broadConsentStatus) {
// bc not asked
return false
}
if (Consent.ConsentProvisionType.PERMIT == broadConsentStatus || Consent.ConsentProvisionType.PERMIT == genomDeSequencingStatus) return true
return false
}
fun embedBroadConsentResources(mtbFile: Mtb, broadConsent: Bundle) {
for (entry in broadConsent.getEntry()) {
val resource = entry.getResource()
if (resource is Consent) {
// since jackson convertValue does not work here,
// we need another step to back to string, before we convert to object map
val asJsonString = fhirContext.newJsonParser().encodeResourceToString(resource)
try {
val mapOfJson: HashMap<String?, Any?>? =
objectMapper.readValue<HashMap<String?, Any?>?>(
asJsonString, object : TypeReference<HashMap<String?, Any?>?>() {})
mtbFile.metadata.researchConsents.add(mapOfJson)
} catch (e: JsonProcessingException) {
throw RuntimeException(e)
}
}
}
}
fun addGenomeDbProvisions(mtbFile: Mtb, consentGnomeDe: Bundle) {
for (entry in consentGnomeDe.getEntry()) {
val resource = entry.getResource()
if (resource !is Consent) {
continue
}
// We expect only one provision in collection, therefore get first or none
val provisions = resource.getProvision().getProvision()
if (provisions.isEmpty()) {
continue
}
val provisionComponent: ProvisionComponent = provisions.first()
var provisionCode: String? = null
if (provisionComponent.getCode() != null && !provisionComponent.getCode().isEmpty()) {
val codableConcept: CodeableConcept = provisionComponent.getCode().first()
if (codableConcept.getCoding() != null && !codableConcept.getCoding().isEmpty()) {
provisionCode = codableConcept.getCoding().first().getCode()
}
}
if (provisionCode != null) {
try {
val modelProjectConsentPurpose =
ModelProjectConsentPurpose.forValue(provisionCode)
if (ModelProjectConsentPurpose.SEQUENCING == modelProjectConsentPurpose) {
// CONVENTION: wrapping date is date of SEQUENCING consent
mtbFile.metadata.modelProjectConsent.date = resource.getDateTime()
}
val provision = Provision.builder()
.type(ConsentProvision.valueOf(provisionComponent.getType().name))
.date(provisionComponent.getPeriod().getStart())
.purpose(modelProjectConsentPurpose).build()
mtbFile.metadata.modelProjectConsent.provisions.add(provision)
} catch (ioe: IOException) {
logger.error(
"Provision code '$provisionCode' is unknown and cannot be mapped.",
ioe.toString()
)
}
}
if (!mtbFile.metadata.modelProjectConsent.provisions.isEmpty()) {
mtbFile.metadata.modelProjectConsent.version =
gIcsConfigProperties.genomeDeConsentVersion
}
}
}
/**
* fixme: currently we do not have information about submission type
*/
private fun setGenomDeSubmissionType(mtbFile: Mtb) {
if (appConfigProperties.genomDeTestSubmission) {
// fixme: uncomment when data model is updated
// mtbFile.metadata.type = MvhSubmissionType.Test
logger.info("genomeDe submission mit TEST")
} else {
mtbFile.metadata.type = MvhSubmissionType.INITIAL
}
}
/**
* @param consentBundle consent resource
* @param requestDate date which must be within validation period of provision
* @return type of provision, will be [org.hl7.fhir.r4.model.Consent.ConsentProvisionType.NULL] if none is found.
*/
fun getProvisionTypeByPolicyCode(
consentBundle: Bundle, requestDate: Date?, consentDomain: ConsentDomain
): Consent.ConsentProvisionType {
val code: String?
val system: String?
if (ConsentDomain.BroadConsent == consentDomain) {
code = gIcsConfigProperties.broadConsentPolicyCode
system = gIcsConfigProperties.broadConsentPolicySystem
} else if (ConsentDomain.Modelvorhaben64e == consentDomain) {
code = gIcsConfigProperties.genomeDePolicyCode
system = gIcsConfigProperties.genomeDePolicySystem
} else {
throw NotImplementedException("unknown consent domain " + consentDomain.name)
}
val provisionTypeByPolicyCode = getProvisionTypeByPolicyCode(
consentBundle, code, system, requestDate
)
return provisionTypeByPolicyCode
}
/**
* @param consentBundle consent resource
* @param policyAndProvisionCode policyRule and provision code value
* @param policyAndProvisionSystem policyRule and provision system value
* @param requestDate date which must be within validation period of provision
* @return type of provision, will be [org.hl7.fhir.r4.model.Consent.ConsentProvisionType.NULL] if none is found.
*/
fun getProvisionTypeByPolicyCode(
consentBundle: Bundle,
policyAndProvisionCode: String?,
policyAndProvisionSystem: String?,
requestDate: Date?
): Consent.ConsentProvisionType {
val entriesOfInterest = consentBundle.entry.filter { entry ->
entry.resource.isResource && entry.resource.resourceType == ResourceType.Consent && (entry.resource as Consent).status == ConsentState.ACTIVE && checkCoding(
policyAndProvisionCode,
policyAndProvisionSystem,
(entry.resource as Consent).policyRule.codingFirstRep
) && isIsRequestDateInRange(
requestDate, (entry.resource as Consent).provision.period
)
}.map { consentWithTargetPolicy: BundleEntryComponent ->
val provision = (consentWithTargetPolicy.getResource() as Consent).getProvision()
val provisionComponentByCode =
provision.getProvision().stream().filter { prov: ProvisionComponent? ->
checkCoding(
policyAndProvisionCode,
policyAndProvisionSystem,
prov!!.getCodeFirstRep().getCodingFirstRep()
) && isIsRequestDateInRange(
requestDate, prov.getPeriod()
)
}.findFirst()
if (provisionComponentByCode.isPresent) {
// actual provision we search for
return@map provisionComponentByCode.get().getType()
} else {
if (provision.type != null) return provision.type
}
return Consent.ConsentProvisionType.NULL
}.firstOrNull()
if (entriesOfInterest == null) return Consent.ConsentProvisionType.NULL
return entriesOfInterest
}
fun checkCoding(
researchAllowedPolicyOid: String?, researchAllowedPolicySystem: String?, coding: Coding
): Boolean {
return coding.getSystem() == researchAllowedPolicySystem && (coding.getCode() == researchAllowedPolicyOid)
}
fun isIsRequestDateInRange(requestdate: Date?, provPeriod: Period): Boolean {
val isRequestDateAfterOrEqualStart = provPeriod.getStart().compareTo(requestdate)
val isRequestDateBeforeOrEqualEnd = provPeriod.getEnd().compareTo(requestdate)
return isRequestDateAfterOrEqualStart <= 0 && isRequestDateBeforeOrEqualEnd >= 0
}
}

View File

@ -23,6 +23,7 @@ import com.fasterxml.jackson.databind.ObjectMapper
import de.ukw.ccc.bwhc.dto.MtbFile
import dev.dnpm.etl.processor.*
import dev.dnpm.etl.processor.config.AppConfigProperties
import dev.dnpm.etl.processor.consent.TtpConsentStatus
import dev.dnpm.etl.processor.monitoring.Report
import dev.dnpm.etl.processor.monitoring.Request
import dev.dnpm.etl.processor.monitoring.RequestStatus
@ -34,8 +35,11 @@ import dev.dnpm.etl.processor.pseudonym.pseudonymizeWith
import dev.pcvolkmer.mv64e.mtb.Mtb
import org.apache.commons.codec.binary.Base32
import org.apache.commons.codec.digest.DigestUtils
import org.slf4j.Logger
import org.slf4j.LoggerFactory
import org.springframework.context.ApplicationEventPublisher
import org.springframework.stereotype.Service
import java.lang.RuntimeException
import java.time.Instant
import java.util.*
@ -47,9 +51,11 @@ class RequestProcessor(
private val requestService: RequestService,
private val objectMapper: ObjectMapper,
private val applicationEventPublisher: ApplicationEventPublisher,
private val appConfigProperties: AppConfigProperties
private val appConfigProperties: AppConfigProperties,
private val consentProcessor: ConsentProcessor?
) {
private var logger: Logger = LoggerFactory.getLogger("RequestProcessor")
fun processMtbFile(mtbFile: MtbFile) {
processMtbFile(mtbFile, randomRequestId())
}
@ -66,12 +72,25 @@ class RequestProcessor(
processMtbFile(mtbFile, randomRequestId())
}
fun processMtbFile(mtbFile: Mtb, requestId: RequestId) {
val pid = PatientId(mtbFile.patient.id)
mtbFile pseudonymizeWith pseudonymizeService
mtbFile anonymizeContentWith pseudonymizeService
val request = DnpmV2MtbFileRequest(requestId, transformationService.transform(mtbFile))
saveAndSend(request, pid)
val pid = PatientId(extractPatientIdentifier(mtbFile))
val isConsentOk = consentProcessor != null &&
consentProcessor.consentGatedCheckAndTryEmbedding(mtbFile) || consentProcessor == null
if (isConsentOk) {
mtbFile pseudonymizeWith pseudonymizeService
mtbFile anonymizeContentWith pseudonymizeService
val request = DnpmV2MtbFileRequest(requestId, transformationService.transform(mtbFile))
saveAndSend(request, pid)
} else {
logger.warn("consent check failed file will not be processed further!")
applicationEventPublisher.publishEvent(
ResponseEvent(
requestId, Instant.now(), RequestStatus.NO_CONSENT
)
)
}
}
private fun <T> saveAndSend(request: MtbFileRequest<T>, pid: PatientId) {
@ -89,9 +108,7 @@ class RequestProcessor(
if (appConfigProperties.duplicationDetection && isDuplication(request)) {
applicationEventPublisher.publishEvent(
ResponseEvent(
request.requestId,
Instant.now(),
RequestStatus.DUPLICATION
request.requestId, Instant.now(), RequestStatus.DUPLICATION
)
)
return
@ -120,21 +137,31 @@ class RequestProcessor(
val lastMtbFileRequestForPatient =
requestService.lastMtbFileRequestForPatientPseudonym(patientPseudonym)
val isLastRequestDeletion = requestService.isLastRequestWithKnownStatusDeletion(patientPseudonym)
val isLastRequestDeletion =
requestService.isLastRequestWithKnownStatusDeletion(patientPseudonym)
return null != lastMtbFileRequestForPatient
&& !isLastRequestDeletion
&& lastMtbFileRequestForPatient.fingerprint == fingerprint(pseudonymizedMtbFileRequest)
return null != lastMtbFileRequestForPatient && !isLastRequestDeletion && lastMtbFileRequestForPatient.fingerprint == fingerprint(
pseudonymizedMtbFileRequest
)
}
fun processDeletion(patientId: PatientId) {
processDeletion(patientId, randomRequestId())
fun processDeletion(patientId: PatientId, isConsented: TtpConsentStatus) {
processDeletion(patientId, randomRequestId(), isConsented)
}
fun processDeletion(patientId: PatientId, requestId: RequestId) {
fun processDeletion(patientId: PatientId, requestId: RequestId, isConsented: TtpConsentStatus) {
try {
val patientPseudonym = pseudonymizeService.patientPseudonym(patientId)
val requestStatus: RequestStatus = when (isConsented) {
TtpConsentStatus.BROAD_CONSENT_MISSING_OR_REJECTED, TtpConsentStatus.BROAD_CONSENT_MISSING, TtpConsentStatus.BROAD_CONSENT_REJECTED -> RequestStatus.NO_CONSENT
TtpConsentStatus.FAILED_TO_ASK -> RequestStatus.ERROR
TtpConsentStatus.BROAD_CONSENT_GIVEN, TtpConsentStatus.UNKNOWN_CHECK_FILE -> RequestStatus.UNKNOWN
TtpConsentStatus.GENOM_DE_CONSENT_SEQUENCING_PERMIT, TtpConsentStatus.GENOM_DE_CONSENT_MISSING, TtpConsentStatus.GENOM_DE_SEQUENCING_REJECTED -> {
throw RuntimeException("processDelete should never deal with '" + isConsented.name + "' consent status. This is a bug and need to be fixed!")
}
}
requestService.save(
Request(
requestId,
@ -142,7 +169,7 @@ class RequestProcessor(
patientId,
fingerprint(patientPseudonym.value),
RequestType.DELETE,
RequestStatus.UNKNOWN
requestStatus
)
)
@ -150,17 +177,14 @@ class RequestProcessor(
applicationEventPublisher.publishEvent(
ResponseEvent(
requestId,
Instant.now(),
responseStatus.status,
when (responseStatus.status) {
requestId, Instant.now(), responseStatus.status, when (responseStatus.status) {
RequestStatus.WARNING, RequestStatus.ERROR -> Optional.of(responseStatus.body)
else -> Optional.empty()
}
)
)
} catch (e: Exception) {
} catch (_: Exception) {
requestService.save(
Request(
uuid = requestId,
@ -184,10 +208,10 @@ class RequestProcessor(
private fun fingerprint(s: String): Fingerprint {
return Fingerprint(
Base32().encodeAsString(DigestUtils.sha256(s))
.replace("=", "")
.lowercase()
Base32().encodeAsString(DigestUtils.sha256(s)).replace("=", "").lowercase()
)
}
}
private fun extractPatientIdentifier(mtbFile: Mtb): String = mtbFile.patient.id

View File

@ -70,6 +70,12 @@ class ResponseProcessor(
)
}
RequestStatus.NO_CONSENT -> {
it.report = Report(
"Einwilligung Status fehlt, widerrufen oder ungeklärt."
)
}
else -> {
logger.error("Cannot process response: Unknown response!")
return@ifPresentOrElse

View File

@ -19,10 +19,7 @@
package dev.dnpm.etl.processor.web
import dev.dnpm.etl.processor.monitoring.ConnectionCheckResult
import dev.dnpm.etl.processor.monitoring.ConnectionCheckService
import dev.dnpm.etl.processor.monitoring.GPasConnectionCheckService
import dev.dnpm.etl.processor.monitoring.OutputConnectionCheckService
import dev.dnpm.etl.processor.monitoring.*
import dev.dnpm.etl.processor.output.MtbFileSender
import dev.dnpm.etl.processor.pseudonym.Generator
import dev.dnpm.etl.processor.security.Role
@ -61,11 +58,15 @@ class ConfigController(
val gPasConnectionAvailable =
connectionCheckServices.filterIsInstance<GPasConnectionCheckService>().firstOrNull()?.connectionAvailable()
val gIcsConnectionAvailable =
connectionCheckServices.filterIsInstance<GIcsConnectionCheckService>().firstOrNull()?.connectionAvailable()
model.addAttribute("pseudonymGenerator", pseudonymGenerator.javaClass.simpleName)
model.addAttribute("mtbFileSender", mtbFileSender.javaClass.simpleName)
model.addAttribute("mtbFileEndpoint", mtbFileSender.endpoint())
model.addAttribute("outputConnectionAvailable", outputConnectionAvailable)
model.addAttribute("gPasConnectionAvailable", gPasConnectionAvailable)
model.addAttribute("gIcsConnectionAvailable", gIcsConnectionAvailable)
model.addAttribute("tokensEnabled", tokenService != null)
if (tokenService != null) {
model.addAttribute("tokens", tokenService.findAll())
@ -119,6 +120,24 @@ class ConfigController(
return "configs/gPasConnectionAvailable"
}
@GetMapping(params = ["gIcsConnectionAvailable"])
fun gIcsConnectionAvailable(model: Model): String {
val gIcsConnectionAvailable =
connectionCheckServices.filterIsInstance<GIcsConnectionCheckService>().firstOrNull()?.connectionAvailable()
model.addAttribute("mtbFileSender", mtbFileSender.javaClass.simpleName)
model.addAttribute("mtbFileEndpoint", mtbFileSender.endpoint())
model.addAttribute("gIcsConnectionAvailable", gIcsConnectionAvailable)
if (tokenService != null) {
model.addAttribute("tokensEnabled", true)
model.addAttribute("tokens", tokenService.findAll())
} else {
model.addAttribute("tokens", listOf<Token>())
}
return "configs/gIcsConnectionAvailable"
}
@PostMapping(path = ["tokens"])
fun addToken(@ModelAttribute("name") name: String, model: Model): String {
if (tokenService == null) {
@ -190,6 +209,7 @@ class ConfigController(
is ConnectionCheckResult.KafkaConnectionCheckResult -> "output-connection-check"
is ConnectionCheckResult.RestConnectionCheckResult -> "output-connection-check"
is ConnectionCheckResult.GPasConnectionCheckResult -> "gpas-connection-check"
is ConnectionCheckResult.GIcsConnectionCheckResult -> "gics-connection-check"
}
ServerSentEvent.builder<Any>()

View File

@ -16,6 +16,7 @@ spring:
content:
enabled: true
paths: /**/*.js,/**/*.css,/**/*.svg,/**/*.jpeg
app:
isGenomDeTestSubmission: true
server:
forward-headers-strategy: framework

View File

@ -49,6 +49,11 @@
</div>
</section>
<section hx-ext="sse" th:sse-connect="@{/configs/events}">
<div th:insert="~{configs/gIcsConnectionAvailable.html}" th:hx-get="@{/configs?gIcsConnectionAvailable}" hx-trigger="sse:gics-connection-check">
</div>
</section>
<section hx-ext="sse" th:sse-connect="@{/configs/events}">
<div th:insert="~{configs/outputConnectionAvailable.html}" th:hx-get="@{/configs?outputConnectionAvailable}" hx-trigger="sse:output-connection-check">
</div>

View File

@ -0,0 +1,24 @@
<th:block th:if="${gIcsConnectionAvailable == null}">
<h2><span>🟦</span> gICS nicht konfiguriert - Einwilligung wird über Dateiinhalt geprüft</h2>
</th:block>
<th:block th:if="${gIcsConnectionAvailable != null}">
<h2><span th:if="${gIcsConnectionAvailable.available}"></span><span th:if="${not(gIcsConnectionAvailable.available)}"></span> Verbindung zu gICS</h2>
<div>
Stand: <time style="font-weight: bold" th:datetime="${#temporals.formatISO(gIcsConnectionAvailable.timestamp)}" th:text="${#temporals.formatISO(gIcsConnectionAvailable.timestamp)}"></time>
&nbsp;|&nbsp;
Letzte Änderung: <time style="font-weight: bold" th:datetime="${#temporals.formatISO(gIcsConnectionAvailable.lastChange)}" th:text="${#temporals.formatISO(gIcsConnectionAvailable.lastChange)}"></time>
</div>
<div>
<span>Die Verbindung ist aktuell</span>
<strong th:if="${gIcsConnectionAvailable.available}" style="color: green">verfügbar.</strong>
<strong th:if="${not(gIcsConnectionAvailable.available)}" style="color: red">nicht verfügbar.</strong>
</div>
<div class="connection-display border">
<img th:src="@{/server.png}" alt="ETL-Processor" />
<span class="connection" th:classappend="${gIcsConnectionAvailable.available ? 'available' : ''}"></span>
<img th:src="@{/server.png}" alt="gICS" />
<span>ETL-Processor</span>
<span></span>
<span>gICS</span>
</div>
</th:block>

View File

@ -0,0 +1,123 @@
package dev.dnpm.etl.processor.consent;
import static org.assertj.core.api.Assertions.assertThat;
import static org.springframework.test.web.client.match.MockRestRequestMatchers.requestTo;
import static org.springframework.test.web.client.response.MockRestResponseCreators.withSuccess;
import com.fasterxml.jackson.databind.ObjectMapper;
import dev.dnpm.etl.processor.config.AppConfiguration;
import dev.dnpm.etl.processor.config.AppFhirConfig;
import dev.dnpm.etl.processor.config.GIcsConfigProperties;
import java.time.Instant;
import java.util.Date;
import org.hl7.fhir.r4.model.BooleanType;
import org.hl7.fhir.r4.model.Identifier;
import org.hl7.fhir.r4.model.OperationOutcome;
import org.hl7.fhir.r4.model.OperationOutcome.IssueSeverity;
import org.hl7.fhir.r4.model.OperationOutcome.IssueType;
import org.hl7.fhir.r4.model.OperationOutcome.OperationOutcomeIssueComponent;
import org.hl7.fhir.r4.model.Parameters;
import org.hl7.fhir.r4.model.Parameters.ParametersParameterComponent;
import org.hl7.fhir.r4.model.StringType;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.autoconfigure.web.client.RestClientTest;
import org.springframework.http.MediaType;
import org.springframework.test.context.ContextConfiguration;
import org.springframework.test.context.TestPropertySource;
import org.springframework.test.web.client.MockRestServiceServer;
@ContextConfiguration(classes = {AppConfiguration.class, ObjectMapper.class})
@TestPropertySource(properties = {"app.consent.gics.enabled=true",
"app.consent.gics.uri=http://localhost:8090/ttp-fhir/fhir/gics"})
@RestClientTest
public class GicsConsentServiceTest {
public static final String GICS_BASE_URI = "http://localhost:8090/ttp-fhir/fhir/gics";
@Autowired
MockRestServiceServer mockRestServiceServer;
@Autowired
GicsConsentService gicsConsentService;
@Autowired
AppConfiguration appConfiguration;
@Autowired
AppFhirConfig appFhirConfig;
@Autowired
GIcsConfigProperties gIcsConfigProperties;
@BeforeEach
public void setUp() {
mockRestServiceServer = MockRestServiceServer.createServer(appConfiguration.restTemplate());
}
@Test
void getTtpBroadConsentStatus() {
final Parameters responseConsented = new Parameters().addParameter(
new ParametersParameterComponent().setName("consented")
.setValue(new BooleanType().setValue(true)));
mockRestServiceServer.expect(requestTo(
"http://localhost:8090/ttp-fhir/fhir/gics" + GicsConsentService.IS_CONSENTED_ENDPOINT))
.andRespond(withSuccess(appFhirConfig.fhirContext().newJsonParser()
.encodeResourceToString(responseConsented), MediaType.APPLICATION_JSON));
var consentStatus = gicsConsentService.getTtpBroadConsentStatus("123456");
assertThat(consentStatus).isEqualTo(TtpConsentStatus.BROAD_CONSENT_GIVEN);
}
@Test
void consentRevoced() {
final Parameters responseRevoced = new Parameters().addParameter(
new ParametersParameterComponent().setName("consented")
.setValue(new BooleanType().setValue(false)));
mockRestServiceServer.expect(requestTo(
"http://localhost:8090/ttp-fhir/fhir/gics" + GicsConsentService.IS_CONSENTED_ENDPOINT))
.andRespond(withSuccess(
appFhirConfig.fhirContext().newJsonParser().encodeResourceToString(responseRevoced),
MediaType.APPLICATION_JSON));
var consentStatus = gicsConsentService.getTtpBroadConsentStatus("123456");
assertThat(consentStatus).isEqualTo(TtpConsentStatus.BROAD_CONSENT_MISSING_OR_REJECTED);
}
@Test
void gicsParameterInvalid() {
final OperationOutcome responseErrorOutcome = new OperationOutcome().addIssue(
new OperationOutcomeIssueComponent().setSeverity(IssueSeverity.ERROR)
.setCode(IssueType.PROCESSING).setDiagnostics("Invalid policy parameter..."));
mockRestServiceServer.expect(
requestTo(GICS_BASE_URI + GicsConsentService.IS_CONSENTED_ENDPOINT)).andRespond(
withSuccess(appFhirConfig.fhirContext().newJsonParser()
.encodeResourceToString(responseErrorOutcome), MediaType.APPLICATION_JSON));
var consentStatus = gicsConsentService.getTtpBroadConsentStatus("123456");
assertThat(consentStatus).isEqualTo(TtpConsentStatus.FAILED_TO_ASK);
}
@Test
void buildRequestParameterCurrentPolicyStatesForPersonTest() {
String pid = "12345678";
var result = GicsConsentService.buildRequestParameterCurrentPolicyStatesForPerson(
gIcsConfigProperties, pid, Date.from(Instant.now()),
gIcsConfigProperties.getGenomDeConsentDomainName());
assertThat(result.getParameter().size()).as("should contain 3 parameter resources")
.isEqualTo(3);
assertThat(((StringType) result.getParameter("domain").getValue()).getValue()).isEqualTo(
gIcsConfigProperties.getGenomDeConsentDomainName());
assertThat(
((Identifier) result.getParameter("personIdentifier").getValue()).getValue()).isEqualTo(
pid);
}
}

View File

@ -23,6 +23,7 @@ import com.fasterxml.jackson.databind.ObjectMapper
import de.ukw.ccc.bwhc.dto.Consent
import de.ukw.ccc.bwhc.dto.MtbFile
import de.ukw.ccc.bwhc.dto.Patient
import dev.dnpm.etl.processor.consent.TtpConsentStatus
import dev.dnpm.etl.processor.CustomMediaType
import dev.dnpm.etl.processor.services.RequestProcessor
import org.apache.kafka.clients.consumer.ConsumerRecord
@ -34,10 +35,7 @@ import org.junit.jupiter.api.Test
import org.junit.jupiter.api.extension.ExtendWith
import org.mockito.Mock
import org.mockito.junit.jupiter.MockitoExtension
import org.mockito.kotlin.any
import org.mockito.kotlin.anyValueClass
import org.mockito.kotlin.times
import org.mockito.kotlin.verify
import org.mockito.kotlin.*
import java.util.*
@ExtendWith(MockitoExtension::class)
@ -49,7 +47,7 @@ class KafkaInputListenerTest {
@BeforeEach
fun setup(
@Mock requestProcessor: RequestProcessor
@Mock requestProcessor: RequestProcessor,
) {
this.requestProcessor = requestProcessor
this.objectMapper = ObjectMapper()
@ -94,7 +92,10 @@ class KafkaInputListenerTest {
)
)
verify(requestProcessor, times(1)).processDeletion(anyValueClass())
verify(requestProcessor, times(1)).processDeletion(
anyValueClass(),
eq(TtpConsentStatus.UNKNOWN_CHECK_FILE)
)
}
@Test
@ -147,7 +148,8 @@ class KafkaInputListenerTest {
Optional.empty()
)
)
verify(requestProcessor, times(1)).processDeletion(anyValueClass(), anyValueClass())
verify(requestProcessor, times(1)).processDeletion(anyValueClass(), anyValueClass(), eq(
TtpConsentStatus.UNKNOWN_CHECK_FILE))
}
@Test
@ -178,7 +180,8 @@ class KafkaInputListenerTest {
Optional.empty()
)
)
verify(requestProcessor, times(0)).processDeletion(anyValueClass(), anyValueClass())
verify(requestProcessor, times(0)).processDeletion(anyValueClass(), anyValueClass(), eq(
TtpConsentStatus.UNKNOWN_CHECK_FILE))
}
}

View File

@ -21,21 +21,29 @@ package dev.dnpm.etl.processor.input
import com.fasterxml.jackson.databind.ObjectMapper
import de.ukw.ccc.bwhc.dto.*
import de.ukw.ccc.bwhc.dto.Consent.Status
import dev.dnpm.etl.processor.CustomMediaType
import dev.dnpm.etl.processor.consent.ConsentByMtbFile
import dev.dnpm.etl.processor.consent.GicsConsentService
import dev.dnpm.etl.processor.consent.TtpConsentStatus
import dev.dnpm.etl.processor.services.RequestProcessor
import dev.pcvolkmer.mv64e.mtb.Mtb
import org.junit.jupiter.api.BeforeEach
import org.junit.jupiter.api.Nested
import org.junit.jupiter.api.Test
import org.junit.jupiter.api.extension.ExtendWith
import org.junit.jupiter.params.ParameterizedTest
import org.junit.jupiter.params.provider.ValueSource
import org.mockito.Mock
import org.mockito.Mockito.times
import org.mockito.Mockito.verify
import org.mockito.junit.jupiter.MockitoExtension
import org.mockito.kotlin.any
import org.mockito.kotlin.anyValueClass
import org.mockito.kotlin.whenever
import org.springframework.core.io.ClassPathResource
import org.springframework.http.MediaType
import org.springframework.test.context.TestPropertySource
import org.springframework.test.web.servlet.MockMvc
import org.springframework.test.web.servlet.delete
import org.springframework.test.web.servlet.post
@ -53,19 +61,22 @@ class MtbFileRestControllerTest {
private lateinit var requestProcessor: RequestProcessor
@BeforeEach
fun setup(
@Mock requestProcessor: RequestProcessor
) {
this.requestProcessor = requestProcessor
val controller = MtbFileRestController(requestProcessor)
val controller = MtbFileRestController(requestProcessor,
ConsentByMtbFile()
)
this.mockMvc = MockMvcBuilders.standaloneSetup(controller).build()
}
@Test
fun shouldProcessPostRequest() {
mockMvc.post("/mtbfile") {
content = objectMapper.writeValueAsString(bwhcMtbFileContent(Consent.Status.ACTIVE))
content = objectMapper.writeValueAsString(bwhcMtbFileContent(Status.ACTIVE))
contentType = MediaType.APPLICATION_JSON
}.andExpect {
status {
@ -79,7 +90,8 @@ class MtbFileRestControllerTest {
@Test
fun shouldProcessPostRequestWithRejectedConsent() {
mockMvc.post("/mtbfile") {
content = objectMapper.writeValueAsString(bwhcMtbFileContent(Consent.Status.REJECTED))
content =
objectMapper.writeValueAsString(bwhcMtbFileContent(Status.REJECTED))
contentType = MediaType.APPLICATION_JSON
}.andExpect {
status {
@ -87,7 +99,10 @@ class MtbFileRestControllerTest {
}
}
verify(requestProcessor, times(1)).processDeletion(anyValueClass())
verify(requestProcessor, times(1)).processDeletion(
anyValueClass(),
org.mockito.kotlin.eq(TtpConsentStatus.BROAD_CONSENT_MISSING_OR_REJECTED)
)
}
@Test
@ -98,10 +113,100 @@ class MtbFileRestControllerTest {
}
}
verify(requestProcessor, times(1)).processDeletion(anyValueClass())
verify(requestProcessor, times(1)).processDeletion(
anyValueClass(),
org.mockito.kotlin.eq(TtpConsentStatus.UNKNOWN_CHECK_FILE)
)
}
}
@TestPropertySource(
properties = ["app.consent.gics.enabled=true",
"app.consent.gics.gIcsBaseUri=http://localhost:8090/ttp-fhir/fhir/gics"]
)
@Nested
inner class BwhcRequestsCheckConsentViaTtp {
private lateinit var mockMvc: MockMvc
private lateinit var requestProcessor: RequestProcessor
private lateinit var gicsConsentService: GicsConsentService
@BeforeEach
fun setup(
@Mock requestProcessor: RequestProcessor,
@Mock gicsConsentService: GicsConsentService
) {
this.requestProcessor = requestProcessor
val controller = MtbFileRestController(requestProcessor, gicsConsentService)
this.mockMvc = MockMvcBuilders.standaloneSetup(controller).build()
this.gicsConsentService = gicsConsentService
}
@ParameterizedTest
@ValueSource(strings = ["ACTIVE", "REJECTED"])
fun shouldProcessPostRequest(status: String) {
whenever(gicsConsentService.getTtpBroadConsentStatus(any())).thenReturn(TtpConsentStatus.BROAD_CONSENT_GIVEN)
mockMvc.post("/mtbfile") {
content =
objectMapper.writeValueAsString(bwhcMtbFileContent(Status.valueOf(status)))
contentType = MediaType.APPLICATION_JSON
}.andExpect {
status {
isAccepted()
}
}
verify(requestProcessor, times(1)).processMtbFile(any<MtbFile>())
}
@ParameterizedTest
@ValueSource(strings = ["ACTIVE", "REJECTED"])
fun shouldProcessPostRequestWithRejectedConsent(status: String) {
whenever(gicsConsentService.getTtpBroadConsentStatus(any())).thenReturn(TtpConsentStatus.BROAD_CONSENT_MISSING_OR_REJECTED)
mockMvc.post("/mtbfile") {
content =
objectMapper.writeValueAsString(bwhcMtbFileContent(Status.valueOf(status)))
contentType = MediaType.APPLICATION_JSON
}.andExpect {
status {
isAccepted()
}
}
// consent status from ttp should override file consent value
verify(requestProcessor, times(1)).processDeletion(
anyValueClass(),
org.mockito.kotlin.eq(TtpConsentStatus.BROAD_CONSENT_MISSING_OR_REJECTED)
)
}
@Test
fun shouldProcessDeleteRequest() {
mockMvc.delete("/mtbfile/TEST_12345678").andExpect {
status {
isAccepted()
}
}
verify(requestProcessor, times(1)).processDeletion(
anyValueClass(),
org.mockito.kotlin.eq(TtpConsentStatus.UNKNOWN_CHECK_FILE)
)
verify(gicsConsentService, times(0)).getTtpBroadConsentStatus(any())
}
}
@Nested
inner class BwhcRequestsWithAlias {
@ -114,14 +219,16 @@ class MtbFileRestControllerTest {
@Mock requestProcessor: RequestProcessor
) {
this.requestProcessor = requestProcessor
val controller = MtbFileRestController(requestProcessor)
val controller = MtbFileRestController(requestProcessor,
ConsentByMtbFile()
)
this.mockMvc = MockMvcBuilders.standaloneSetup(controller).build()
}
@Test
fun shouldProcessPostRequest() {
mockMvc.post("/mtb") {
content = objectMapper.writeValueAsString(bwhcMtbFileContent(Consent.Status.ACTIVE))
content = objectMapper.writeValueAsString(bwhcMtbFileContent(Status.ACTIVE))
contentType = MediaType.APPLICATION_JSON
}.andExpect {
status {
@ -135,7 +242,8 @@ class MtbFileRestControllerTest {
@Test
fun shouldProcessPostRequestWithRejectedConsent() {
mockMvc.post("/mtb") {
content = objectMapper.writeValueAsString(bwhcMtbFileContent(Consent.Status.REJECTED))
content =
objectMapper.writeValueAsString(bwhcMtbFileContent(Status.REJECTED))
contentType = MediaType.APPLICATION_JSON
}.andExpect {
status {
@ -143,7 +251,11 @@ class MtbFileRestControllerTest {
}
}
verify(requestProcessor, times(1)).processDeletion(anyValueClass())
verify(requestProcessor, times(1)).processDeletion(
anyValueClass(), org.mockito.kotlin.eq(
TtpConsentStatus.BROAD_CONSENT_MISSING_OR_REJECTED
)
)
}
@Test
@ -154,7 +266,11 @@ class MtbFileRestControllerTest {
}
}
verify(requestProcessor, times(1)).processDeletion(anyValueClass())
verify(requestProcessor, times(1)).processDeletion(
anyValueClass(), org.mockito.kotlin.eq(
TtpConsentStatus.UNKNOWN_CHECK_FILE
)
)
}
}
@ -167,16 +283,21 @@ class MtbFileRestControllerTest {
@BeforeEach
fun setup(
@Mock requestProcessor: RequestProcessor
@Mock requestProcessor: RequestProcessor,
@Mock gicsConsentService: GicsConsentService
) {
this.requestProcessor = requestProcessor
val controller = MtbFileRestController(requestProcessor)
val controller = MtbFileRestController(requestProcessor,
gicsConsentService
)
this.mockMvc = MockMvcBuilders.standaloneSetup(controller).build()
}
@Test
fun shouldRespondPostRequest() {
val mtbFileContent = ClassPathResource("mv64e-mtb-fake-patient.json").inputStream.readAllBytes().toString(Charsets.UTF_8)
val mtbFileContent =
ClassPathResource("mv64e-mtb-fake-patient.json").inputStream.readAllBytes()
.toString(Charsets.UTF_8)
mockMvc.post("/mtb") {
content = mtbFileContent
@ -193,7 +314,7 @@ class MtbFileRestControllerTest {
}
companion object {
fun bwhcMtbFileContent(consentStatus: Consent.Status) = MtbFile.builder()
fun bwhcMtbFileContent(consentStatus: Status) = MtbFile.builder()
.withPatient(
Patient.builder()
.withId("TEST_12345678")

View File

@ -19,11 +19,18 @@
package dev.dnpm.etl.processor.pseudonym
import ca.uhn.fhir.context.FhirContext
import com.fasterxml.jackson.databind.ObjectMapper
import de.ukw.ccc.bwhc.dto.*
import de.ukw.ccc.bwhc.dto.Patient
import dev.dnpm.etl.processor.config.AppConfigProperties
import dev.dnpm.etl.processor.config.GIcsConfigProperties
import dev.dnpm.etl.processor.config.JacksonConfig
import dev.dnpm.etl.processor.services.ConsentProcessor
import dev.dnpm.etl.processor.services.ConsentProcessorTest
import dev.pcvolkmer.mv64e.mtb.*
import org.assertj.core.api.Assertions.assertThat
import org.hl7.fhir.r4.model.Bundle
import org.junit.jupiter.api.Nested
import org.junit.jupiter.api.Test
import org.junit.jupiter.api.assertThrows
@ -39,6 +46,9 @@ import java.util.*
@ExtendWith(MockitoExtension::class)
class ExtensionsTest {
fun getObjectMapper(): ObjectMapper {
return JacksonConfig().objectMapper()
}
@Nested
inner class UsingBwhcDatamodel {
@ -46,13 +56,14 @@ class ExtensionsTest {
val FAKE_MTB_FILE_PATH = "fake_MTBFile.json"
val CLEAN_PATIENT_ID = "5dad2f0b-49c6-47d8-a952-7b9e9e0f7549"
private fun fakeMtbFile(): MtbFile {
val mtbFile = ClassPathResource(FAKE_MTB_FILE_PATH).inputStream
return ObjectMapper().readValue(mtbFile, MtbFile::class.java)
return getObjectMapper().readValue(mtbFile, MtbFile::class.java)
}
private fun MtbFile.serialized(): String {
return ObjectMapper().writeValueAsString(this)
return getObjectMapper().writeValueAsString(this)
}
@Test
@ -86,7 +97,9 @@ class ExtensionsTest {
mtbFile.pseudonymizeWith(pseudonymizeService)
mtbFile.anonymizeContentWith(pseudonymizeService)
val pattern = "\"[a-f0-9]{8}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{12}\"".toRegex().toPattern()
val pattern =
"\"[a-f0-9]{8}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{12}\"".toRegex()
.toPattern()
val matcher = pattern.matcher(mtbFile.serialized())
assertThrows<IllegalStateException> {
@ -207,15 +220,15 @@ class ExtensionsTest {
inner class UsingDnpmV2Datamodel {
val FAKE_MTB_FILE_PATH = "mv64e-mtb-fake-patient.json"
val CLEAN_PATIENT_ID = "aca5a971-28be-4089-8128-0036a4fe6f1a"
val CLEAN_PATIENT_ID = "644bae7a-56f6-4ee8-b02f-c532e65af5b1"
private fun fakeMtbFile(): Mtb {
val mtbFile = ClassPathResource(FAKE_MTB_FILE_PATH).inputStream
return ObjectMapper().readValue(mtbFile, Mtb::class.java)
return getObjectMapper().readValue(mtbFile, Mtb::class.java)
}
private fun Mtb.serialized(): String {
return ObjectMapper().writeValueAsString(this)
return getObjectMapper().writeValueAsString(this)
}
@Test
@ -226,6 +239,8 @@ class ExtensionsTest {
}.whenever(pseudonymizeService).patientPseudonym(anyValueClass())
val mtbFile = fakeMtbFile()
mtbFile.ensureMetaDataIsInitialized()
addConsentData(mtbFile)
mtbFile.pseudonymizeWith(pseudonymizeService)
@ -233,6 +248,25 @@ class ExtensionsTest {
assertThat(mtbFile.serialized()).doesNotContain(CLEAN_PATIENT_ID)
}
private fun addConsentData(mtbFile: Mtb) {
val gIcsConfigProperties = GIcsConfigProperties("", "", "", true)
val appConfigProperties = AppConfigProperties(null, emptyList())
val bundle = Bundle()
val dummyConsent = ConsentProcessorTest.getDummyGenomDeConsent()
dummyConsent.patient.reference = "Patient/$CLEAN_PATIENT_ID"
bundle.addEntry().resource = dummyConsent
ConsentProcessor(
appConfigProperties,
gIcsConfigProperties,
JacksonConfig().objectMapper(),
FhirContext.forR4(),
null
).embedBroadConsentResources(mtbFile, bundle)
}
@Test
fun shouldNotThrowExceptionOnNullValues(@Mock pseudonymizeService: PseudonymizeService) {
doAnswer {

View File

@ -0,0 +1,168 @@
package dev.dnpm.etl.processor.services
import ca.uhn.fhir.context.FhirContext
import com.fasterxml.jackson.databind.ObjectMapper
import dev.dnpm.etl.processor.config.AppConfigProperties
import dev.dnpm.etl.processor.config.GIcsConfigProperties
import dev.dnpm.etl.processor.config.JacksonConfig
import dev.dnpm.etl.processor.consent.GicsConsentService
import dev.pcvolkmer.mv64e.mtb.*
import org.assertj.core.api.Assertions.assertThat
import org.hl7.fhir.r4.model.Bundle
import org.hl7.fhir.r4.model.CodeableConcept
import org.hl7.fhir.r4.model.Coding
import org.hl7.fhir.r4.model.Consent
import org.junit.jupiter.api.BeforeEach
import org.junit.jupiter.api.Test
import org.junit.jupiter.api.extension.ExtendWith
import org.junit.jupiter.params.ParameterizedTest
import org.junit.jupiter.params.provider.CsvSource
import org.mockito.Mock
import org.mockito.junit.jupiter.MockitoExtension
import org.mockito.kotlin.any
import org.mockito.kotlin.doAnswer
import org.mockito.kotlin.whenever
import org.springframework.core.io.ClassPathResource
import java.io.IOException
import java.io.InputStream
import java.time.Instant
import java.time.OffsetDateTime
import java.util.*
@ExtendWith(MockitoExtension::class)
class ConsentProcessorTest {
private lateinit var appConfigProperties: AppConfigProperties
private lateinit var gicsConsentService: GicsConsentService
private lateinit var objectMapper: ObjectMapper
private lateinit var gIcsConfigProperties: GIcsConfigProperties
private lateinit var fhirContext: FhirContext
private lateinit var consentProcessor: ConsentProcessor
@BeforeEach
fun setups(
@Mock gicsConsentService: GicsConsentService,
) {
this.gIcsConfigProperties = GIcsConfigProperties(null, null, null, true)
val jacksonConfig = JacksonConfig()
this.objectMapper = jacksonConfig.objectMapper()
this.fhirContext = JacksonConfig.fhirContext()
this.gicsConsentService = gicsConsentService
this.appConfigProperties = AppConfigProperties(null, emptyList())
this.consentProcessor =
ConsentProcessor(
appConfigProperties,
gIcsConfigProperties,
objectMapper,
fhirContext,
gicsConsentService
)
}
@Test
fun consentOk() {
assertThat(consentProcessor.toString()).isNotNull
// prep gICS response
doAnswer { getDummyBroadConsentBundle() }.whenever(gicsConsentService)
.getBroadConsent(any(), any())
doAnswer { Bundle() }.whenever(gicsConsentService)
.getGenomDeConsent(any(), any())
val inputMtb = Mtb.builder()
.patient(Patient.builder().id("d611d429-5003-11f0-a144-661e92ac9503").build()).build()
val checkResult = consentProcessor.consentGatedCheckAndTryEmbedding(inputMtb)
assertThat(checkResult).isTrue
assertThat(inputMtb.metadata.researchConsents).hasSize(13)
}
companion object {
fun getDummyGenomDeConsent(): Consent {
val consent = Consent()
consent.id = "consent 1 id"
consent.patient.reference = "Patient/1234-pat1"
consent.provision.setType(
Consent.ConsentProvisionType.fromCode(
"deny"
)
)
consent.provision.period.start =
Date.from(Instant.parse("2025-06-23T00:00:00.00Z"))
consent.provision.period.end =
Date.from(Instant.parse("3000-01-01T00:00:00.00Z"))
val addProvision1 = consent.provision.addProvision()
addProvision1.setType(Consent.ConsentProvisionType.fromCode("permit"))
addProvision1.period.start = Date.from(Instant.parse("2025-06-23T00:00:00.00Z"))
addProvision1.period.end = Date.from(Instant.parse("3000-01-01T00:00:00.00Z"))
addProvision1.code.addLast(
CodeableConcept(
Coding(
"https://ths-greifswald.de/fhir/CodeSystem/gics/Policy/GenomDE_MV",
"Teilnahme",
"Teilnahme am Modellvorhaben und Einwilligung zur Genomsequenzierung"
)
)
)
val addProvision2 = consent.provision.addProvision()
addProvision2.setType(Consent.ConsentProvisionType.fromCode("deny"))
addProvision2.period.start = Date.from(Instant.parse("2025-06-23T00:00:00.00Z"))
addProvision2.period.end = Date.from(Instant.parse("3000-01-01T00:00:00.00Z"))
addProvision2.code.addLast(
CodeableConcept(
Coding(
"https://ths-greifswald.de/fhir/CodeSystem/gics/Policy/GenomDE_MV",
"Rekontaktierung",
"Re-Identifizierung meiner Daten über die Vertrauensstelle beim Robert Koch-Institut und in die erneute Kontaktaufnahme durch meine behandelnde Ärztin oder meinen behandelnden Arzt"
)
)
)
return consent
}
}
@ParameterizedTest
@CsvSource(
"2.16.840.1.113883.3.1937.777.24.5.3.8,urn:oid:2.16.840.1.113883.3.1937.777.24.5.3,2025-07-23T00:00:00+02:00,PERMIT,expect permit",
"2.16.840.1.113883.3.1937.777.24.5.3.8,urn:oid:2.16.840.1.113883.3.1937.777.24.5.3,2025-06-23T00:00:00+02:00,PERMIT,expect permit date is exactly on start",
"2.16.840.1.113883.3.1937.777.24.5.3.8,urn:oid:2.16.840.1.113883.3.1937.777.24.5.3,2055-06-23T00:00:00+02:00,PERMIT,expect permit date is exactly on end",
"2.16.840.1.113883.3.1937.777.24.5.3.8,urn:oid:2.16.840.1.113883.3.1937.777.24.5.3,2021-06-23T00:00:00+02:00,NULL,date is before start",
"2.16.840.1.113883.3.1937.777.24.5.3.8,urn:oid:2.16.840.1.113883.3.1937.777.24.5.3,2060-06-23T00:00:00+02:00,NULL,date is after end",
"2.16.840.1.113883.3.1937.777.24.5.3.8,XXXX,2025-07-23T00:00:00+02:00,NULL,system not found - therefore expect NULL",
"2.16.840.1.113883.3.1937.777.24.5.3.27,urn:oid:2.16.840.1.113883.3.1937.777.24.5.3,2025-07-23T00:00:00+02:00,DENY,provision is denied"
)
fun getProvisionTypeByPolicyCode(
code: String?, system: String?, timeStamp: String, expected: String?,
desc: String?
) {
val testData = getDummyBroadConsentBundle()
val requestDate = Date.from(OffsetDateTime.parse(timeStamp).toInstant())
val result: Consent.ConsentProvisionType =
consentProcessor.getProvisionTypeByPolicyCode(testData, code, system, requestDate)
assertThat(result).isNotNull()
assertThat(result).`as`(desc)
.isEqualTo(Consent.ConsentProvisionType.valueOf(expected!!))
}
fun getDummyBroadConsentBundle(): Bundle {
val bundle: InputStream?
try {
bundle = ClassPathResource(
"fake_broadConsent_gics_response_permit.json"
).getInputStream()
} catch (e: IOException) {
throw RuntimeException(e)
}
return FhirContext.forR4().newJsonParser()
.parseResource<Bundle>(Bundle::class.java, bundle)
}
}

View File

@ -25,6 +25,8 @@ import dev.dnpm.etl.processor.Fingerprint
import dev.dnpm.etl.processor.PatientId
import dev.dnpm.etl.processor.PatientPseudonym
import dev.dnpm.etl.processor.config.AppConfigProperties
import dev.dnpm.etl.processor.consent.GicsConsentService
import dev.dnpm.etl.processor.consent.TtpConsentStatus
import dev.dnpm.etl.processor.monitoring.Request
import dev.dnpm.etl.processor.monitoring.RequestStatus
import dev.dnpm.etl.processor.monitoring.RequestType
@ -58,7 +60,7 @@ class RequestProcessorTest {
private lateinit var requestService: RequestService
private lateinit var applicationEventPublisher: ApplicationEventPublisher
private lateinit var appConfigProperties: AppConfigProperties
private lateinit var consentProcessor: ConsentProcessor
private lateinit var requestProcessor: RequestProcessor
@BeforeEach
@ -67,7 +69,8 @@ class RequestProcessorTest {
@Mock transformationService: TransformationService,
@Mock sender: RestMtbFileSender,
@Mock requestService: RequestService,
@Mock applicationEventPublisher: ApplicationEventPublisher
@Mock applicationEventPublisher: ApplicationEventPublisher,
@Mock consentProcessor: ConsentProcessor
) {
this.pseudonymizeService = pseudonymizeService
this.transformationService = transformationService
@ -75,6 +78,7 @@ class RequestProcessorTest {
this.requestService = requestService
this.applicationEventPublisher = applicationEventPublisher
this.appConfigProperties = AppConfigProperties(null)
this.consentProcessor = consentProcessor
val objectMapper = ObjectMapper()
@ -85,7 +89,8 @@ class RequestProcessorTest {
requestService,
objectMapper,
applicationEventPublisher,
appConfigProperties
appConfigProperties,
consentProcessor
)
}
@ -343,7 +348,10 @@ class RequestProcessorTest {
MtbFileSender.Response(status = RequestStatus.UNKNOWN)
}.whenever(sender).send(any<DeleteRequest>())
this.requestProcessor.processDeletion(TEST_PATIENT_ID)
this.requestProcessor.processDeletion(
TEST_PATIENT_ID,
isConsented = TtpConsentStatus.UNKNOWN_CHECK_FILE
)
val requestCaptor = argumentCaptor<Request>()
verify(requestService, times(1)).save(requestCaptor.capture())
@ -361,7 +369,10 @@ class RequestProcessorTest {
MtbFileSender.Response(status = RequestStatus.SUCCESS)
}.whenever(sender).send(any<DeleteRequest>())
this.requestProcessor.processDeletion(TEST_PATIENT_ID)
this.requestProcessor.processDeletion(
TEST_PATIENT_ID,
isConsented = TtpConsentStatus.UNKNOWN_CHECK_FILE
)
val eventCaptor = argumentCaptor<ResponseEvent>()
verify(applicationEventPublisher, times(1)).publishEvent(eventCaptor.capture())
@ -379,7 +390,10 @@ class RequestProcessorTest {
MtbFileSender.Response(status = RequestStatus.ERROR)
}.whenever(sender).send(any<DeleteRequest>())
this.requestProcessor.processDeletion(TEST_PATIENT_ID)
this.requestProcessor.processDeletion(
TEST_PATIENT_ID,
isConsented = TtpConsentStatus.UNKNOWN_CHECK_FILE
)
val eventCaptor = argumentCaptor<ResponseEvent>()
verify(applicationEventPublisher, times(1)).publishEvent(eventCaptor.capture())
@ -391,7 +405,10 @@ class RequestProcessorTest {
fun testShouldSendDeleteRequestWithPseudonymErrorAndSaveErrorRequestStatus() {
doThrow(RuntimeException()).whenever(pseudonymizeService).patientPseudonym(anyValueClass())
this.requestProcessor.processDeletion(TEST_PATIENT_ID)
this.requestProcessor.processDeletion(
TEST_PATIENT_ID,
isConsented = TtpConsentStatus.UNKNOWN_CHECK_FILE
)
val requestCaptor = argumentCaptor<Request>()
verify(requestService, times(1)).save(requestCaptor.capture())

View File

@ -19,14 +19,23 @@
package dev.dnpm.etl.processor.services
import com.fasterxml.jackson.databind.ObjectMapper
import de.ukw.ccc.bwhc.dto.Consent
import de.ukw.ccc.bwhc.dto.Diagnosis
import de.ukw.ccc.bwhc.dto.Icd10
import de.ukw.ccc.bwhc.dto.MtbFile
import dev.dnpm.etl.processor.config.JacksonConfig
import dev.pcvolkmer.mv64e.mtb.ConsentProvision
import dev.pcvolkmer.mv64e.mtb.ModelProjectConsent
import dev.pcvolkmer.mv64e.mtb.ModelProjectConsentPurpose
import org.assertj.core.api.Assertions.assertThat
import org.junit.jupiter.api.BeforeEach
import org.junit.jupiter.api.Test
import dev.pcvolkmer.mv64e.mtb.Mtb
import dev.pcvolkmer.mv64e.mtb.MvhMetadata
import dev.pcvolkmer.mv64e.mtb.Provision
import org.hl7.fhir.instance.model.api.IBaseResource
import java.time.Instant
import java.util.Date
class TransformationServiceTest {
@ -35,7 +44,7 @@ class TransformationServiceTest {
@BeforeEach
fun setup() {
this.service = TransformationService(
ObjectMapper(), listOf(
JacksonConfig().objectMapper(), listOf(
Transformation.of("consent.status") from Consent.Status.ACTIVE to Consent.Status.REJECTED,
Transformation.of("diagnoses[*].icd10.version") from "2013" to "2014",
)
@ -92,4 +101,59 @@ class TransformationServiceTest {
assertThat(actual.consent.status).isEqualTo(Consent.Status.REJECTED)
}
@Test
fun shouldTransformConsentValues() {
val mtbFile = MtbFile.builder().withDiagnoses(
listOf(
Diagnosis.builder().withId("1234").withIcd10(Icd10("F79.9").also {
it.version = "2013"
}).build(),
Diagnosis.builder().withId("5678").withIcd10(Icd10("F79.8").also {
it.version = "2019"
}).build()
)
).build()
val actual = this.service.transform(mtbFile)
assertThat(actual).isNotNull
assertThat(actual.diagnoses[0].icd10.code).isEqualTo("F79.9")
assertThat(actual.diagnoses[0].icd10.version).isEqualTo("2014")
assertThat(actual.diagnoses[1].icd10.code).isEqualTo("F79.8")
assertThat(actual.diagnoses[1].icd10.version).isEqualTo("2019")
}
@Test
fun shouldTransformConsent() {
val mvhMetadata = MvhMetadata.builder().transferTan("transfertan12345").build()
assertThat(mvhMetadata).isNotNull
mvhMetadata.modelProjectConsent =
ModelProjectConsent.builder().date(Date.from(Instant.parse("2025-06-23T00:00:00.00Z")))
.version("1").provisions(
listOf(
Provision.builder().type(ConsentProvision.PERMIT)
.purpose(ModelProjectConsentPurpose.SEQUENCING)
.date(Date.from(Instant.parse("2025-06-23T00:00:00.00Z"))).build(),
Provision.builder().type(ConsentProvision.PERMIT)
.purpose(ModelProjectConsentPurpose.REIDENTIFICATION)
.date(Date.from(Instant.parse("2025-06-23T00:00:00.00Z"))).build(),
Provision.builder().type(ConsentProvision.DENY)
.purpose(ModelProjectConsentPurpose.CASE_IDENTIFICATION)
.date(Date.from(Instant.parse("2025-06-23T00:00:00.00Z"))).build()
)
).build()
val consent = ConsentProcessorTest.getDummyGenomDeConsent()
mvhMetadata.researchConsents = mutableListOf()
mvhMetadata.researchConsents.add(mapOf(consent.id to consent as IBaseResource))
val mtbFile = Mtb.builder().metadata(mvhMetadata).build()
val transformed = service.transform(mtbFile)
assertThat(transformed.metadata.modelProjectConsent.date).isNotNull
}
}

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

View File

@ -0,0 +1,333 @@
{
"resourceType": "Bundle",
"type": "collection",
"entry": [
{
"fullUrl": "http://127.0.0.1:8090/ttp-fhir/fhir/gics/Consent/24673204-50e1-11f0-a144-661e92ac9503",
"resource": {
"resourceType": "Consent",
"id": "24673204-50e1-11f0-a144-661e92ac9503",
"meta": {
"lastUpdated": "2025-06-24T11:58:27.178+02:00",
"profile": [
"http://fhir.de/ConsentManagement/StructureDefinition/Consent"
]
},
"extension": [
{
"url": "http://fhir.de/ConsentManagement/StructureDefinition/DomainReference",
"extension": [
{
"url": "domain",
"valueReference": {
"reference": "ResearchStudy/ef86d80e-50e0-11f0-a144-661e92ac9503"
}
},
{
"url": "status",
"valueCoding": {
"system": "http://hl7.org/fhir/publication-status",
"code": "active"
}
}
]
}
],
"status": "active",
"scope": {
"coding": [
{
"system": "http://terminology.hl7.org/CodeSystem/consentscope",
"code": "research"
}
]
},
"category": [
{
"coding": [
{
"system": "http://loinc.org",
"code": "59284-0"
}
]
},
{
"coding": [
{
"system": "http://fhir.de/ConsentManagement/CodeSystem/ResultType",
"code": "policy"
}
]
}
],
"patient": {
"reference": "Patient/2466d49b-50e1-11f0-a144-661e92ac9503",
"display": "Patienten-ID 999999"
},
"dateTime": "2025-06-24T00:00:00+02:00",
"organization": [
{
"display": "GenomDE_MV"
}
],
"sourceReference": {
"reference": "QuestionnaireResponse/24670c77-50e1-11f0-a144-661e92ac9503"
},
"policyRule": {
"coding": [
{
"system": "https://ths-greifswald.de/fhir/CodeSystem/gics/Policy/GenomDE_MV",
"code": "Teilnahme",
"display": "Teilnahme am Modellvorhaben und Einwilligung zur Genomsequenzierung"
}
]
},
"provision": {
"type": "deny",
"period": {
"start": "2025-06-24T00:00:00+02:00",
"end": "3000-01-01T00:00:00+01:00"
},
"provision": [
{
"type": "deny",
"period": {
"start": "2025-06-24T00:00:00+02:00",
"end": "3000-01-01T00:00:00+01:00"
},
"code": [
{
"coding": [
{
"system": "https://ths-greifswald.de/fhir/CodeSystem/gics/Policy/GenomDE_MV",
"code": "Teilnahme",
"display": "Teilnahme am Modellvorhaben und Einwilligung zur Genomsequenzierung"
}
]
}
]
}
]
}
}
},
{
"fullUrl": "http://127.0.0.1:8090/ttp-fhir/fhir/gics/Consent/24673913-50e1-11f0-a144-661e92ac9503",
"resource": {
"resourceType": "Consent",
"id": "24673913-50e1-11f0-a144-661e92ac9503",
"meta": {
"lastUpdated": "2025-06-24T11:58:27.194+02:00",
"profile": [
"http://fhir.de/ConsentManagement/StructureDefinition/Consent"
]
},
"extension": [
{
"url": "http://fhir.de/ConsentManagement/StructureDefinition/DomainReference",
"extension": [
{
"url": "domain",
"valueReference": {
"reference": "ResearchStudy/ef86d80e-50e0-11f0-a144-661e92ac9503"
}
},
{
"url": "status",
"valueCoding": {
"system": "http://hl7.org/fhir/publication-status",
"code": "active"
}
}
]
}
],
"status": "active",
"scope": {
"coding": [
{
"system": "http://terminology.hl7.org/CodeSystem/consentscope",
"code": "research"
}
]
},
"category": [
{
"coding": [
{
"system": "http://loinc.org",
"code": "59284-0"
}
]
},
{
"coding": [
{
"system": "http://fhir.de/ConsentManagement/CodeSystem/ResultType",
"code": "policy"
}
]
}
],
"patient": {
"reference": "Patient/2466d49b-50e1-11f0-a144-661e92ac9503",
"display": "Patienten-ID 999999"
},
"dateTime": "2025-06-24T00:00:00+02:00",
"organization": [
{
"display": "GenomDE_MV"
}
],
"sourceReference": {
"reference": "QuestionnaireResponse/24670c77-50e1-11f0-a144-661e92ac9503"
},
"policyRule": {
"coding": [
{
"system": "https://ths-greifswald.de/fhir/CodeSystem/gics/Policy/GenomDE_MV",
"code": "Fallidentifizierung",
"display": "Fallidentifizierung zum fachlichen Austausch unter Behandelnden"
}
]
},
"provision": {
"type": "deny",
"period": {
"start": "2025-06-24T00:00:00+02:00",
"end": "3000-01-01T00:00:00+01:00"
},
"provision": [
{
"type": "deny",
"period": {
"start": "2025-06-24T00:00:00+02:00",
"end": "3000-01-01T00:00:00+01:00"
},
"code": [
{
"coding": [
{
"system": "https://ths-greifswald.de/fhir/CodeSystem/gics/Policy/GenomDE_MV",
"code": "Fallidentifizierung",
"display": "Fallidentifizierung zum fachlichen Austausch unter Behandelnden"
}
]
}
]
}
]
}
}
},
{
"fullUrl": "http://127.0.0.1:8090/ttp-fhir/fhir/gics/Consent/24673da9-50e1-11f0-a144-661e92ac9503",
"resource": {
"resourceType": "Consent",
"id": "24673da9-50e1-11f0-a144-661e92ac9503",
"meta": {
"lastUpdated": "2025-06-24T11:58:27.211+02:00",
"profile": [
"http://fhir.de/ConsentManagement/StructureDefinition/Consent"
]
},
"extension": [
{
"url": "http://fhir.de/ConsentManagement/StructureDefinition/DomainReference",
"extension": [
{
"url": "domain",
"valueReference": {
"reference": "ResearchStudy/ef86d80e-50e0-11f0-a144-661e92ac9503"
}
},
{
"url": "status",
"valueCoding": {
"system": "http://hl7.org/fhir/publication-status",
"code": "active"
}
}
]
}
],
"status": "active",
"scope": {
"coding": [
{
"system": "http://terminology.hl7.org/CodeSystem/consentscope",
"code": "research"
}
]
},
"category": [
{
"coding": [
{
"system": "http://loinc.org",
"code": "59284-0"
}
]
},
{
"coding": [
{
"system": "http://fhir.de/ConsentManagement/CodeSystem/ResultType",
"code": "policy"
}
]
}
],
"patient": {
"reference": "Patient/2466d49b-50e1-11f0-a144-661e92ac9503",
"display": "Patienten-ID 999999"
},
"dateTime": "2025-06-24T00:00:00+02:00",
"organization": [
{
"display": "GenomDE_MV"
}
],
"sourceReference": {
"reference": "QuestionnaireResponse/24670c77-50e1-11f0-a144-661e92ac9503"
},
"policyRule": {
"coding": [
{
"system": "https://ths-greifswald.de/fhir/CodeSystem/gics/Policy/GenomDE_MV",
"code": "Rekontaktierung",
"display": "Re-Identifizierung meiner Daten über die Vertrauensstelle beim Robert Koch-Institut und in die erneute Kontaktaufnahme durch meine behandelnde Ärztin oder meinen behandelnden Arzt"
}
]
},
"provision": {
"type": "deny",
"period": {
"start": "2025-06-24T00:00:00+02:00",
"end": "3000-01-01T00:00:00+01:00"
},
"provision": [
{
"type": "deny",
"period": {
"start": "2025-06-24T00:00:00+02:00",
"end": "3000-01-01T00:00:00+01:00"
},
"code": [
{
"coding": [
{
"system": "https://ths-greifswald.de/fhir/CodeSystem/gics/Policy/GenomDE_MV",
"code": "Rekontaktierung",
"display": "Re-Identifizierung meiner Daten über die Vertrauensstelle beim Robert Koch-Institut und in die erneute Kontaktaufnahme durch meine behandelnde Ärztin oder meinen behandelnden Arzt"
}
]
}
]
}
]
}
}
}
]
}

View File

@ -0,0 +1,333 @@
{
"resourceType": "Bundle",
"type": "collection",
"entry": [
{
"fullUrl": "http://127.0.0.1:8090/ttp-fhir/fhir/gics/Consent/121a8368-50e1-11f0-a144-661e92ac9503",
"resource": {
"resourceType": "Consent",
"id": "121a8368-50e1-11f0-a144-661e92ac9503",
"meta": {
"lastUpdated": "2025-06-24T11:55:42.079+02:00",
"profile": [
"http://fhir.de/ConsentManagement/StructureDefinition/Consent"
]
},
"extension": [
{
"url": "http://fhir.de/ConsentManagement/StructureDefinition/DomainReference",
"extension": [
{
"url": "domain",
"valueReference": {
"reference": "ResearchStudy/ef86d80e-50e0-11f0-a144-661e92ac9503"
}
},
{
"url": "status",
"valueCoding": {
"system": "http://hl7.org/fhir/publication-status",
"code": "active"
}
}
]
}
],
"status": "active",
"scope": {
"coding": [
{
"system": "http://terminology.hl7.org/CodeSystem/consentscope",
"code": "research"
}
]
},
"category": [
{
"coding": [
{
"system": "http://loinc.org",
"code": "59284-0"
}
]
},
{
"coding": [
{
"system": "http://fhir.de/ConsentManagement/CodeSystem/ResultType",
"code": "policy"
}
]
}
],
"patient": {
"reference": "Patient/12194791-50e1-11f0-a144-661e92ac9503",
"display": "Patienten-ID 12345678"
},
"dateTime": "2025-06-24T00:00:00+02:00",
"organization": [
{
"display": "GenomDE_MV"
}
],
"sourceReference": {
"reference": "QuestionnaireResponse/1219ca42-50e1-11f0-a144-661e92ac9503"
},
"policyRule": {
"coding": [
{
"system": "https://ths-greifswald.de/fhir/CodeSystem/gics/Policy/GenomDE_MV",
"code": "Teilnahme",
"display": "Teilnahme am Modellvorhaben und Einwilligung zur Genomsequenzierung"
}
]
},
"provision": {
"type": "deny",
"period": {
"start": "2025-06-24T00:00:00+02:00",
"end": "3000-01-01T00:00:00+01:00"
},
"provision": [
{
"type": "permit",
"period": {
"start": "2025-06-24T00:00:00+02:00",
"end": "3000-01-01T00:00:00+01:00"
},
"code": [
{
"coding": [
{
"system": "https://ths-greifswald.de/fhir/CodeSystem/gics/Policy/GenomDE_MV",
"code": "Teilnahme",
"display": "Teilnahme am Modellvorhaben und Einwilligung zur Genomsequenzierung"
}
]
}
]
}
]
}
}
},
{
"fullUrl": "http://127.0.0.1:8090/ttp-fhir/fhir/gics/Consent/121aad40-50e1-11f0-a144-661e92ac9503",
"resource": {
"resourceType": "Consent",
"id": "121aad40-50e1-11f0-a144-661e92ac9503",
"meta": {
"lastUpdated": "2025-06-24T11:55:42.096+02:00",
"profile": [
"http://fhir.de/ConsentManagement/StructureDefinition/Consent"
]
},
"extension": [
{
"url": "http://fhir.de/ConsentManagement/StructureDefinition/DomainReference",
"extension": [
{
"url": "domain",
"valueReference": {
"reference": "ResearchStudy/ef86d80e-50e0-11f0-a144-661e92ac9503"
}
},
{
"url": "status",
"valueCoding": {
"system": "http://hl7.org/fhir/publication-status",
"code": "active"
}
}
]
}
],
"status": "active",
"scope": {
"coding": [
{
"system": "http://terminology.hl7.org/CodeSystem/consentscope",
"code": "research"
}
]
},
"category": [
{
"coding": [
{
"system": "http://loinc.org",
"code": "59284-0"
}
]
},
{
"coding": [
{
"system": "http://fhir.de/ConsentManagement/CodeSystem/ResultType",
"code": "policy"
}
]
}
],
"patient": {
"reference": "Patient/12194791-50e1-11f0-a144-661e92ac9503",
"display": "Patienten-ID 12345678"
},
"dateTime": "2025-06-24T00:00:00+02:00",
"organization": [
{
"display": "GenomDE_MV"
}
],
"sourceReference": {
"reference": "QuestionnaireResponse/1219ca42-50e1-11f0-a144-661e92ac9503"
},
"policyRule": {
"coding": [
{
"system": "https://ths-greifswald.de/fhir/CodeSystem/gics/Policy/GenomDE_MV",
"code": "Fallidentifizierung",
"display": "Fallidentifizierung zum fachlichen Austausch unter Behandelnden"
}
]
},
"provision": {
"type": "deny",
"period": {
"start": "2025-06-24T00:00:00+02:00",
"end": "3000-01-01T00:00:00+01:00"
},
"provision": [
{
"type": "permit",
"period": {
"start": "2025-06-24T00:00:00+02:00",
"end": "3000-01-01T00:00:00+01:00"
},
"code": [
{
"coding": [
{
"system": "https://ths-greifswald.de/fhir/CodeSystem/gics/Policy/GenomDE_MV",
"code": "Fallidentifizierung",
"display": "Fallidentifizierung zum fachlichen Austausch unter Behandelnden"
}
]
}
]
}
]
}
}
},
{
"fullUrl": "http://127.0.0.1:8090/ttp-fhir/fhir/gics/Consent/121ac5f8-50e1-11f0-a144-661e92ac9503",
"resource": {
"resourceType": "Consent",
"id": "121ac5f8-50e1-11f0-a144-661e92ac9503",
"meta": {
"lastUpdated": "2025-06-24T11:55:42.110+02:00",
"profile": [
"http://fhir.de/ConsentManagement/StructureDefinition/Consent"
]
},
"extension": [
{
"url": "http://fhir.de/ConsentManagement/StructureDefinition/DomainReference",
"extension": [
{
"url": "domain",
"valueReference": {
"reference": "ResearchStudy/ef86d80e-50e0-11f0-a144-661e92ac9503"
}
},
{
"url": "status",
"valueCoding": {
"system": "http://hl7.org/fhir/publication-status",
"code": "active"
}
}
]
}
],
"status": "active",
"scope": {
"coding": [
{
"system": "http://terminology.hl7.org/CodeSystem/consentscope",
"code": "research"
}
]
},
"category": [
{
"coding": [
{
"system": "http://loinc.org",
"code": "59284-0"
}
]
},
{
"coding": [
{
"system": "http://fhir.de/ConsentManagement/CodeSystem/ResultType",
"code": "policy"
}
]
}
],
"patient": {
"reference": "Patient/12194791-50e1-11f0-a144-661e92ac9503",
"display": "Patienten-ID 12345678"
},
"dateTime": "2025-06-24T00:00:00+02:00",
"organization": [
{
"display": "GenomDE_MV"
}
],
"sourceReference": {
"reference": "QuestionnaireResponse/1219ca42-50e1-11f0-a144-661e92ac9503"
},
"policyRule": {
"coding": [
{
"system": "https://ths-greifswald.de/fhir/CodeSystem/gics/Policy/GenomDE_MV",
"code": "Rekontaktierung",
"display": "Re-Identifizierung meiner Daten über die Vertrauensstelle beim Robert Koch-Institut und in die erneute Kontaktaufnahme durch meine behandelnde Ärztin oder meinen behandelnden Arzt"
}
]
},
"provision": {
"type": "deny",
"period": {
"start": "2025-06-24T00:00:00+02:00",
"end": "3000-01-01T00:00:00+01:00"
},
"provision": [
{
"type": "permit",
"period": {
"start": "2025-06-24T00:00:00+02:00",
"end": "3000-01-01T00:00:00+01:00"
},
"code": [
{
"coding": [
{
"system": "https://ths-greifswald.de/fhir/CodeSystem/gics/Policy/GenomDE_MV",
"code": "Rekontaktierung",
"display": "Re-Identifizierung meiner Daten über die Vertrauensstelle beim Robert Koch-Institut und in die erneute Kontaktaufnahme durch meine behandelnde Ärztin oder meinen behandelnden Arzt"
}
]
}
]
}
]
}
}
}
]
}

File diff suppressed because it is too large Load Diff