mirror of
https://github.com/pcvolkmer/mv64e-etl-processor
synced 2025-09-13 09:02:50 +00:00
refactor: remove obsolete bwHC data model V1.0 (#129)
This commit is contained in:
46
README.md
46
README.md
@@ -1,6 +1,6 @@
|
|||||||
# ETL-Processor for DNPM:DIP [](https://github.com/pcvolkmer/etl-processor/actions/workflows/test.yml)
|
# ETL-Processor for DNPM:DIP [](https://github.com/pcvolkmer/etl-processor/actions/workflows/test.yml)
|
||||||
|
|
||||||
Diese Anwendung versendet ein bwHC-MTB-File im bwHC-Datenmodell 1.0 an DNPM:DIP und pseudonymisiert
|
Diese Anwendung versendet ein MTB-File im DNPM-Datenmodell 2.1 an DNPM:DIP und pseudonymisiert
|
||||||
die Patienten-ID.
|
die Patienten-ID.
|
||||||
|
|
||||||
## Einordnung innerhalb einer DNPM-ETL-Strecke
|
## Einordnung innerhalb einer DNPM-ETL-Strecke
|
||||||
@@ -8,7 +8,7 @@ die Patienten-ID.
|
|||||||
Diese Anwendung erlaubt das Entgegennehmen von HTTP/REST-Anfragen aus dem Onkostar-Plugin *
|
Diese Anwendung erlaubt das Entgegennehmen von HTTP/REST-Anfragen aus dem Onkostar-Plugin *
|
||||||
*[onkostar-plugin-dnpmexport](https://github.com/CCC-MF/onkostar-plugin-dnpmexport)**.
|
*[onkostar-plugin-dnpmexport](https://github.com/CCC-MF/onkostar-plugin-dnpmexport)**.
|
||||||
|
|
||||||
Der Inhalt einer Anfrage, wenn ein bwHC-MTBFile, wird pseudonymisiert und auf Duplikate geprüft.
|
Der Inhalt einer Anfrage, wenn ein MTB-File, wird pseudonymisiert und auf Duplikate geprüft.
|
||||||
Duplikate werden verworfen, Änderungen werden weitergeleitet.
|
Duplikate werden verworfen, Änderungen werden weitergeleitet.
|
||||||
|
|
||||||
Löschanfragen werden immer als Löschanfrage an DNPM:DIP weitergeleitet.
|
Löschanfragen werden immer als Löschanfrage an DNPM:DIP weitergeleitet.
|
||||||
@@ -72,24 +72,9 @@ Siehe hierzu auch: https://github.com/CCC-MF/kafka-to-bwhc
|
|||||||
|
|
||||||
## Konfiguration
|
## Konfiguration
|
||||||
|
|
||||||
### 🔥 Wichtige Änderungen in Version 0.10
|
### 🔥 Wichtige Änderungen in Version 0.11
|
||||||
|
|
||||||
Ab Version 0.10 wird [DNPM:DIP](https://github.com/dnpm-dip) unterstützt und als Standardendpunkt
|
Ab Version 0.11 wird ausschließlich [DNPM:DIP](https://github.com/dnpm-dip) unterstützt.
|
||||||
verwendet.
|
|
||||||
Soll noch das alte bwHC-Backend verwendet werden, so ist die Umgebungsvariable `APP_REST_IS_BWHC`
|
|
||||||
auf `true` zu setzen.
|
|
||||||
|
|
||||||
### 🔥 Breaking Changes nach Version 0.10
|
|
||||||
|
|
||||||
In Versionen des ETL-Processors **nach Version 0.10** werden die folgenden Konfigurationsoptionen
|
|
||||||
entfernt:
|
|
||||||
|
|
||||||
* `APP_KAFKA_TOPIC`: Nutzen Sie nun die Konfigurationsoption `APP_KAFKA_OUTPUT_TOPIC`
|
|
||||||
* `APP_KAFKA_RESPONSE_TOPIC`: Nutzen Sie nun die Konfigurationsoption
|
|
||||||
`APP_KAFKA_OUTPUT_RESPONSE_TOPIC`
|
|
||||||
|
|
||||||
Der Pfad zum Versenden von MTB-Daten ist nun offiziell `/mtb`.
|
|
||||||
In Versionen **nach Version 0.10** wird die Unterstützung des Pfads `/mtbfile` entfernt.
|
|
||||||
|
|
||||||
### Pseudonymisierung der Patienten-ID
|
### Pseudonymisierung der Patienten-ID
|
||||||
|
|
||||||
@@ -316,18 +301,15 @@ Werden sowohl REST als auch Kafka-Endpunkt konfiguriert, wird nur der REST-Endpu
|
|||||||
|
|
||||||
#### REST
|
#### REST
|
||||||
|
|
||||||
Folgende Umgebungsvariablen müssen gesetzt sein, damit ein bwHC-MTB-File an DNPM:DIP gesendet wird:
|
Folgende Umgebungsvariablen müssen gesetzt sein, damit ein MTB-File an DNPM:DIP gesendet wird:
|
||||||
|
|
||||||
* `APP_REST_URI`: URI der zu benutzenden API der Backend-Instanz. Zum Beispiel:
|
* `APP_REST_URI`: URI der zu benutzenden API der Backend-Instanz. Zum Beispiel `http://localhost:9000/api`
|
||||||
* `http://localhost:9000/bwhc/etl/api` für **bwHC Backend**
|
|
||||||
* `http://localhost:9000/api` für **dnpm:dip**
|
|
||||||
* `APP_REST_USERNAME`: Basic-Auth-Benutzername für den REST-Endpunkt
|
* `APP_REST_USERNAME`: Basic-Auth-Benutzername für den REST-Endpunkt
|
||||||
* `APP_REST_PASSWORD`: Basic-Auth-Passwort für den REST-Endpunkt
|
* `APP_REST_PASSWORD`: Basic-Auth-Passwort für den REST-Endpunkt
|
||||||
* `APP_REST_IS_BWHC`: `true` für **bwHC Backend**, weglassen oder `false` für **dnpm:dip**
|
|
||||||
|
|
||||||
#### Kafka-Topics
|
#### Kafka-Topics
|
||||||
|
|
||||||
Folgende Umgebungsvariablen müssen gesetzt sein, damit ein bwHC-MTB-File an ein Kafka-Topic
|
Folgende Umgebungsvariablen müssen gesetzt sein, damit ein MTB-File an ein Kafka-Topic
|
||||||
übermittelt wird:
|
übermittelt wird:
|
||||||
|
|
||||||
* `APP_KAFKA_OUTPUT_TOPIC`: Zu verwendendes Topic zum Versenden von Anfragen.
|
* `APP_KAFKA_OUTPUT_TOPIC`: Zu verwendendes Topic zum Versenden von Anfragen.
|
||||||
@@ -402,19 +384,7 @@ verwenden möchten.
|
|||||||
|
|
||||||
### Antworten und Statusauswertung
|
### Antworten und Statusauswertung
|
||||||
|
|
||||||
Anfragen an das bwHC-Backend aus Versionen bis 0.9.x wurden wie folgt behandelt:
|
Seit Version 0.10 wird die Issue-Liste der Antwort verwendet und die darion enthaltene höchste
|
||||||
|
|
||||||
| HTTP-Response | Status |
|
|
||||||
|----------------|-----------|
|
|
||||||
| `HTTP 200` | `SUCCESS` |
|
|
||||||
| `HTTP 201` | `WARNING` |
|
|
||||||
| `HTTP 400-...` | `ERROR` |
|
|
||||||
|
|
||||||
Dies konnte dazu führen, dass zwar mit einem `HTTP 201` geantwortet wurde, aber dennoch in der
|
|
||||||
Issue-Liste die
|
|
||||||
Severity `error` aufgetaucht ist.
|
|
||||||
|
|
||||||
Ab Version 0.10 wird die Issue-Liste der Antwort verwendet und die darion enthaltene höchste
|
|
||||||
Severity-Stufe als Ergebnis verwendet.
|
Severity-Stufe als Ergebnis verwendet.
|
||||||
|
|
||||||
| Höchste Severity | Status |
|
| Höchste Severity | Status |
|
||||||
|
@@ -16,7 +16,6 @@ group = "dev.dnpm"
|
|||||||
version = "0.11.0-SNAPSHOT"
|
version = "0.11.0-SNAPSHOT"
|
||||||
|
|
||||||
var versions = mapOf(
|
var versions = mapOf(
|
||||||
"bwhc-dto-java" to "0.4.0",
|
|
||||||
"mtb-dto" to "0.1.0-SNAPSHOT",
|
"mtb-dto" to "0.1.0-SNAPSHOT",
|
||||||
"hapi-fhir" to "7.6.1",
|
"hapi-fhir" to "7.6.1",
|
||||||
"mockito-kotlin" to "5.4.0",
|
"mockito-kotlin" to "5.4.0",
|
||||||
@@ -81,7 +80,6 @@ dependencies {
|
|||||||
implementation("org.flywaydb:flyway-mysql")
|
implementation("org.flywaydb:flyway-mysql")
|
||||||
implementation("commons-codec:commons-codec")
|
implementation("commons-codec:commons-codec")
|
||||||
implementation("io.projectreactor.kotlin:reactor-kotlin-extensions")
|
implementation("io.projectreactor.kotlin:reactor-kotlin-extensions")
|
||||||
implementation("de.ukw.ccc:bwhc-dto-java:${versions["bwhc-dto-java"]}")
|
|
||||||
implementation("dev.pcvolkmer.mv64e:mtb-dto:${versions["mtb-dto"]}") { isChanging = true }
|
implementation("dev.pcvolkmer.mv64e:mtb-dto:${versions["mtb-dto"]}") { isChanging = true }
|
||||||
implementation("ca.uhn.hapi.fhir:hapi-fhir-base:${versions["hapi-fhir"]}")
|
implementation("ca.uhn.hapi.fhir:hapi-fhir-base:${versions["hapi-fhir"]}")
|
||||||
implementation("ca.uhn.hapi.fhir:hapi-fhir-structures-r4:${versions["hapi-fhir"]}")
|
implementation("ca.uhn.hapi.fhir:hapi-fhir-structures-r4:${versions["hapi-fhir"]}")
|
||||||
|
@@ -20,11 +20,11 @@
|
|||||||
package dev.dnpm.etl.processor
|
package dev.dnpm.etl.processor
|
||||||
|
|
||||||
import com.fasterxml.jackson.databind.ObjectMapper
|
import com.fasterxml.jackson.databind.ObjectMapper
|
||||||
import de.ukw.ccc.bwhc.dto.*
|
|
||||||
import dev.dnpm.etl.processor.monitoring.RequestRepository
|
import dev.dnpm.etl.processor.monitoring.RequestRepository
|
||||||
import dev.dnpm.etl.processor.monitoring.RequestStatus
|
import dev.dnpm.etl.processor.monitoring.RequestStatus
|
||||||
import dev.dnpm.etl.processor.output.BwhcV1MtbFileRequest
|
import dev.dnpm.etl.processor.output.DnpmV2MtbFileRequest
|
||||||
import dev.dnpm.etl.processor.output.MtbFileSender
|
import dev.dnpm.etl.processor.output.MtbFileSender
|
||||||
|
import dev.pcvolkmer.mv64e.mtb.*
|
||||||
import org.assertj.core.api.Assertions.assertThat
|
import org.assertj.core.api.Assertions.assertThat
|
||||||
import org.junit.jupiter.api.BeforeEach
|
import org.junit.jupiter.api.BeforeEach
|
||||||
import org.junit.jupiter.api.Nested
|
import org.junit.jupiter.api.Nested
|
||||||
@@ -69,7 +69,7 @@ class EtlProcessorApplicationTests : AbstractTestcontainerTest() {
|
|||||||
properties = [
|
properties = [
|
||||||
"app.pseudonymize.generator=buildin",
|
"app.pseudonymize.generator=buildin",
|
||||||
"app.consent.service=none",
|
"app.consent.service=none",
|
||||||
"app.transformations[0].path=diagnoses[*].icd10.version",
|
"app.transformations[0].path=diagnoses[*].code.version",
|
||||||
"app.transformations[0].from=2013",
|
"app.transformations[0].from=2013",
|
||||||
"app.transformations[0].to=2014",
|
"app.transformations[0].to=2014",
|
||||||
]
|
]
|
||||||
@@ -94,36 +94,21 @@ class EtlProcessorApplicationTests : AbstractTestcontainerTest() {
|
|||||||
fun mtbFileIsTransformed() {
|
fun mtbFileIsTransformed() {
|
||||||
doAnswer {
|
doAnswer {
|
||||||
MtbFileSender.Response(RequestStatus.SUCCESS)
|
MtbFileSender.Response(RequestStatus.SUCCESS)
|
||||||
}.whenever(mtbFileSender).send(any<BwhcV1MtbFileRequest>())
|
}.whenever(mtbFileSender).send(any<DnpmV2MtbFileRequest>())
|
||||||
|
|
||||||
val mtbFile = MtbFile.builder()
|
val mtbFile = Mtb.builder()
|
||||||
.withPatient(
|
.patient(
|
||||||
Patient.builder()
|
Patient.builder()
|
||||||
.withId("TEST_12345678")
|
.id("TEST_12345678")
|
||||||
.withBirthDate("2000-08-08")
|
|
||||||
.withGender(Patient.Gender.MALE)
|
|
||||||
.build()
|
.build()
|
||||||
)
|
)
|
||||||
.withConsent(
|
.diagnoses(
|
||||||
Consent.builder()
|
|
||||||
.withId("1")
|
|
||||||
.withStatus(Consent.Status.ACTIVE)
|
|
||||||
.withPatient("TEST_12345678")
|
|
||||||
.build()
|
|
||||||
)
|
|
||||||
.withEpisode(
|
|
||||||
Episode.builder()
|
|
||||||
.withId("1")
|
|
||||||
.withPatient("TEST_12345678")
|
|
||||||
.withPeriod(PeriodStart("2023-08-08"))
|
|
||||||
.build()
|
|
||||||
)
|
|
||||||
.withDiagnoses(
|
|
||||||
listOf(
|
listOf(
|
||||||
Diagnosis.builder()
|
MtbDiagnosis.builder()
|
||||||
.withId("1234")
|
.id("1234")
|
||||||
.withIcd10(Icd10.builder().withCode("F79.9").withVersion("2013").build())
|
.patient(Reference.builder().id("TEST_12345678").build())
|
||||||
.build()
|
.code(Coding.builder().code("F79.9").version("2013").build())
|
||||||
|
.build(),
|
||||||
)
|
)
|
||||||
)
|
)
|
||||||
.build()
|
.build()
|
||||||
@@ -137,10 +122,10 @@ class EtlProcessorApplicationTests : AbstractTestcontainerTest() {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
val captor = argumentCaptor<BwhcV1MtbFileRequest>()
|
val captor = argumentCaptor<DnpmV2MtbFileRequest>()
|
||||||
verify(mtbFileSender).send(captor.capture())
|
verify(mtbFileSender).send(captor.capture())
|
||||||
assertThat(captor.firstValue.content.diagnoses).hasSize(1).allMatch { diagnosis ->
|
assertThat(captor.firstValue.content.diagnoses).hasSize(1).allMatch { diagnosis ->
|
||||||
diagnosis.icd10.version == "2014"
|
diagnosis.code.version == "2014"
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
@@ -20,7 +20,6 @@
|
|||||||
package dev.dnpm.etl.processor.input
|
package dev.dnpm.etl.processor.input
|
||||||
|
|
||||||
import com.fasterxml.jackson.databind.ObjectMapper
|
import com.fasterxml.jackson.databind.ObjectMapper
|
||||||
import de.ukw.ccc.bwhc.dto.*
|
|
||||||
import dev.dnpm.etl.processor.anyValueClass
|
import dev.dnpm.etl.processor.anyValueClass
|
||||||
import dev.dnpm.etl.processor.config.AppSecurityConfiguration
|
import dev.dnpm.etl.processor.config.AppSecurityConfiguration
|
||||||
import dev.dnpm.etl.processor.consent.ConsentByMtbFile
|
import dev.dnpm.etl.processor.consent.ConsentByMtbFile
|
||||||
@@ -29,6 +28,7 @@ import dev.dnpm.etl.processor.consent.IGetConsent
|
|||||||
import dev.dnpm.etl.processor.security.TokenRepository
|
import dev.dnpm.etl.processor.security.TokenRepository
|
||||||
import dev.dnpm.etl.processor.security.UserRoleRepository
|
import dev.dnpm.etl.processor.security.UserRoleRepository
|
||||||
import dev.dnpm.etl.processor.services.RequestProcessor
|
import dev.dnpm.etl.processor.services.RequestProcessor
|
||||||
|
import dev.pcvolkmer.mv64e.mtb.*
|
||||||
import org.junit.jupiter.api.BeforeEach
|
import org.junit.jupiter.api.BeforeEach
|
||||||
import org.junit.jupiter.api.Nested
|
import org.junit.jupiter.api.Nested
|
||||||
import org.junit.jupiter.api.Test
|
import org.junit.jupiter.api.Test
|
||||||
@@ -48,6 +48,8 @@ import org.springframework.test.context.junit.jupiter.SpringExtension
|
|||||||
import org.springframework.test.web.servlet.MockMvc
|
import org.springframework.test.web.servlet.MockMvc
|
||||||
import org.springframework.test.web.servlet.delete
|
import org.springframework.test.web.servlet.delete
|
||||||
import org.springframework.test.web.servlet.post
|
import org.springframework.test.web.servlet.post
|
||||||
|
import java.time.Instant
|
||||||
|
import java.util.*
|
||||||
|
|
||||||
@WebMvcTest(controllers = [MtbFileRestController::class])
|
@WebMvcTest(controllers = [MtbFileRestController::class])
|
||||||
@ExtendWith(value = [MockitoExtension::class, SpringExtension::class])
|
@ExtendWith(value = [MockitoExtension::class, SpringExtension::class])
|
||||||
@@ -93,7 +95,7 @@ class MtbFileRestControllerTest {
|
|||||||
status { isAccepted() }
|
status { isAccepted() }
|
||||||
}
|
}
|
||||||
|
|
||||||
verify(requestProcessor, times(1)).processMtbFile(any<MtbFile>())
|
verify(requestProcessor, times(1)).processMtbFile(any<Mtb>())
|
||||||
}
|
}
|
||||||
|
|
||||||
@Test
|
@Test
|
||||||
@@ -106,7 +108,7 @@ class MtbFileRestControllerTest {
|
|||||||
status { isAccepted() }
|
status { isAccepted() }
|
||||||
}
|
}
|
||||||
|
|
||||||
verify(requestProcessor, times(1)).processMtbFile(any<MtbFile>())
|
verify(requestProcessor, times(1)).processMtbFile(any<Mtb>())
|
||||||
}
|
}
|
||||||
|
|
||||||
@Test
|
@Test
|
||||||
@@ -119,7 +121,7 @@ class MtbFileRestControllerTest {
|
|||||||
status { isUnauthorized() }
|
status { isUnauthorized() }
|
||||||
}
|
}
|
||||||
|
|
||||||
verify(requestProcessor, never()).processMtbFile(any<MtbFile>())
|
verify(requestProcessor, never()).processMtbFile(any<Mtb>())
|
||||||
}
|
}
|
||||||
|
|
||||||
@Test
|
@Test
|
||||||
@@ -132,7 +134,7 @@ class MtbFileRestControllerTest {
|
|||||||
status { isForbidden() }
|
status { isForbidden() }
|
||||||
}
|
}
|
||||||
|
|
||||||
verify(requestProcessor, never()).processMtbFile(any<MtbFile>())
|
verify(requestProcessor, never()).processMtbFile(any<Mtb>())
|
||||||
}
|
}
|
||||||
|
|
||||||
@Test
|
@Test
|
||||||
@@ -180,7 +182,7 @@ class MtbFileRestControllerTest {
|
|||||||
status { isAccepted() }
|
status { isAccepted() }
|
||||||
}
|
}
|
||||||
|
|
||||||
verify(requestProcessor, times(1)).processMtbFile(any<MtbFile>())
|
verify(requestProcessor, times(1)).processMtbFile(any<Mtb>())
|
||||||
}
|
}
|
||||||
|
|
||||||
@Test
|
@Test
|
||||||
@@ -193,33 +195,26 @@ class MtbFileRestControllerTest {
|
|||||||
status { isAccepted() }
|
status { isAccepted() }
|
||||||
}
|
}
|
||||||
|
|
||||||
verify(requestProcessor, times(1)).processMtbFile(any<MtbFile>())
|
verify(requestProcessor, times(1)).processMtbFile(any<Mtb>())
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
companion object {
|
companion object {
|
||||||
|
|
||||||
val mtbFile: MtbFile = MtbFile.builder()
|
val mtbFile = Mtb.builder()
|
||||||
.withPatient(
|
.patient(
|
||||||
Patient.builder()
|
Patient.builder()
|
||||||
.withId("PID")
|
.id("PID")
|
||||||
.withBirthDate("2000-08-08")
|
|
||||||
.withGender(Patient.Gender.MALE)
|
|
||||||
.build()
|
.build()
|
||||||
)
|
)
|
||||||
.withConsent(
|
.episodesOfCare(
|
||||||
Consent.builder()
|
listOf(
|
||||||
.withId("1")
|
MtbEpisodeOfCare.builder()
|
||||||
.withStatus(Consent.Status.ACTIVE)
|
.id("1")
|
||||||
.withPatient("PID")
|
.patient(Reference.builder().id("PID").build())
|
||||||
.build()
|
.period(PeriodDate.builder().start(Date.from(Instant.parse("2023-08-08T02:00:00.00Z"))).build())
|
||||||
)
|
.build()
|
||||||
.withEpisode(
|
)
|
||||||
Episode.builder()
|
|
||||||
.withId("1")
|
|
||||||
.withPatient("PID")
|
|
||||||
.withPeriod(PeriodStart("2023-08-08"))
|
|
||||||
.build()
|
|
||||||
)
|
)
|
||||||
.build()
|
.build()
|
||||||
|
|
||||||
|
@@ -24,7 +24,6 @@ import org.springframework.boot.context.properties.ConfigurationProperties
|
|||||||
|
|
||||||
@ConfigurationProperties(AppConfigProperties.NAME)
|
@ConfigurationProperties(AppConfigProperties.NAME)
|
||||||
data class AppConfigProperties(
|
data class AppConfigProperties(
|
||||||
var bwhcUri: String?,
|
|
||||||
var transformations: List<TransformationProperties> = listOf(),
|
var transformations: List<TransformationProperties> = listOf(),
|
||||||
var maxRetryAttempts: Int = 3,
|
var maxRetryAttempts: Int = 3,
|
||||||
var duplicationDetection: Boolean = true,
|
var duplicationDetection: Boolean = true,
|
||||||
@@ -128,8 +127,7 @@ data class GIcsConfigProperties(
|
|||||||
data class RestTargetProperties(
|
data class RestTargetProperties(
|
||||||
val uri: String?,
|
val uri: String?,
|
||||||
val username: String?,
|
val username: String?,
|
||||||
val password: String?,
|
val password: String?
|
||||||
val isBwhc: Boolean = false,
|
|
||||||
) {
|
) {
|
||||||
companion object {
|
companion object {
|
||||||
const val NAME = "app.rest"
|
const val NAME = "app.rest"
|
||||||
|
@@ -24,7 +24,6 @@ import dev.dnpm.etl.processor.monitoring.ConnectionCheckService
|
|||||||
import dev.dnpm.etl.processor.monitoring.ReportService
|
import dev.dnpm.etl.processor.monitoring.ReportService
|
||||||
import dev.dnpm.etl.processor.monitoring.RestConnectionCheckService
|
import dev.dnpm.etl.processor.monitoring.RestConnectionCheckService
|
||||||
import dev.dnpm.etl.processor.output.MtbFileSender
|
import dev.dnpm.etl.processor.output.MtbFileSender
|
||||||
import dev.dnpm.etl.processor.output.RestBwhcMtbFileSender
|
|
||||||
import dev.dnpm.etl.processor.output.RestDipMtbFileSender
|
import dev.dnpm.etl.processor.output.RestDipMtbFileSender
|
||||||
import org.slf4j.LoggerFactory
|
import org.slf4j.LoggerFactory
|
||||||
import org.springframework.boot.autoconfigure.condition.ConditionalOnMissingBean
|
import org.springframework.boot.autoconfigure.condition.ConditionalOnMissingBean
|
||||||
@@ -57,11 +56,6 @@ class AppRestConfiguration {
|
|||||||
retryTemplate: RetryTemplate,
|
retryTemplate: RetryTemplate,
|
||||||
reportService: ReportService,
|
reportService: ReportService,
|
||||||
): MtbFileSender {
|
): MtbFileSender {
|
||||||
if (restTargetProperties.isBwhc) {
|
|
||||||
logger.info("Selected 'RestBwhcMtbFileSender'")
|
|
||||||
return RestBwhcMtbFileSender(restTemplate, restTargetProperties, retryTemplate, reportService)
|
|
||||||
}
|
|
||||||
|
|
||||||
logger.info("Selected 'RestDipMtbFileSender'")
|
logger.info("Selected 'RestDipMtbFileSender'")
|
||||||
return RestDipMtbFileSender(restTemplate, restTargetProperties, retryTemplate, reportService)
|
return RestDipMtbFileSender(restTemplate, restTargetProperties, retryTemplate, reportService)
|
||||||
}
|
}
|
||||||
|
@@ -20,13 +20,13 @@
|
|||||||
package dev.dnpm.etl.processor.input
|
package dev.dnpm.etl.processor.input
|
||||||
|
|
||||||
import com.fasterxml.jackson.databind.ObjectMapper
|
import com.fasterxml.jackson.databind.ObjectMapper
|
||||||
import de.ukw.ccc.bwhc.dto.Consent
|
|
||||||
import de.ukw.ccc.bwhc.dto.MtbFile
|
|
||||||
import dev.dnpm.etl.processor.CustomMediaType
|
import dev.dnpm.etl.processor.CustomMediaType
|
||||||
import dev.dnpm.etl.processor.PatientId
|
import dev.dnpm.etl.processor.PatientId
|
||||||
import dev.dnpm.etl.processor.RequestId
|
import dev.dnpm.etl.processor.RequestId
|
||||||
import dev.dnpm.etl.processor.consent.TtpConsentStatus
|
import dev.dnpm.etl.processor.consent.TtpConsentStatus
|
||||||
import dev.dnpm.etl.processor.services.RequestProcessor
|
import dev.dnpm.etl.processor.services.RequestProcessor
|
||||||
|
import dev.pcvolkmer.mv64e.mtb.ConsentProvision
|
||||||
|
import dev.pcvolkmer.mv64e.mtb.Mtb
|
||||||
import org.apache.kafka.clients.consumer.ConsumerRecord
|
import org.apache.kafka.clients.consumer.ConsumerRecord
|
||||||
import org.slf4j.LoggerFactory
|
import org.slf4j.LoggerFactory
|
||||||
import org.springframework.http.MediaType
|
import org.springframework.http.MediaType
|
||||||
@@ -40,7 +40,7 @@ class KafkaInputListener(
|
|||||||
|
|
||||||
override fun onMessage(record: ConsumerRecord<String, String>) {
|
override fun onMessage(record: ConsumerRecord<String, String>) {
|
||||||
when (guessMimeType(record)) {
|
when (guessMimeType(record)) {
|
||||||
MediaType.APPLICATION_JSON_VALUE -> handleBwhcMessage(record)
|
MediaType.APPLICATION_JSON_VALUE -> handleDnpmV2Message(record)
|
||||||
CustomMediaType.APPLICATION_VND_DNPM_V2_MTB_JSON_VALUE -> handleDnpmV2Message(record)
|
CustomMediaType.APPLICATION_VND_DNPM_V2_MTB_JSON_VALUE -> handleDnpmV2Message(record)
|
||||||
else -> {
|
else -> {
|
||||||
/* ignore other messages */
|
/* ignore other messages */
|
||||||
@@ -57,8 +57,11 @@ class KafkaInputListener(
|
|||||||
return record.headers().headers("contentType")?.firstOrNull()?.value().contentToString()
|
return record.headers().headers("contentType")?.firstOrNull()?.value().contentToString()
|
||||||
}
|
}
|
||||||
|
|
||||||
private fun handleBwhcMessage(record: ConsumerRecord<String, String>) {
|
private fun handleDnpmV2Message(record: ConsumerRecord<String, String>) {
|
||||||
val mtbFile = objectMapper.readValue(record.value(), MtbFile::class.java)
|
// Do not handle DNPM-V2 for now
|
||||||
|
logger.warn("Ignoring MTB File in DNPM V2 format: Not implemented yet")
|
||||||
|
|
||||||
|
val mtbFile = objectMapper.readValue(record.value(), Mtb::class.java)
|
||||||
val patientId = PatientId(mtbFile.patient.id)
|
val patientId = PatientId(mtbFile.patient.id)
|
||||||
val firstRequestIdHeader = record.headers().headers("requestId")?.firstOrNull()
|
val firstRequestIdHeader = record.headers().headers("requestId")?.firstOrNull()
|
||||||
val requestId = if (null != firstRequestIdHeader) {
|
val requestId = if (null != firstRequestIdHeader) {
|
||||||
@@ -67,7 +70,8 @@ class KafkaInputListener(
|
|||||||
RequestId("")
|
RequestId("")
|
||||||
}
|
}
|
||||||
|
|
||||||
if (mtbFile.consent.status == Consent.Status.ACTIVE) {
|
// TODO: Use MV Consent for now - needs to be replaced with proper consent evaluation
|
||||||
|
if (mtbFile.metadata.modelProjectConsent.provisions.filter { it.type == ConsentProvision.PERMIT }.isNotEmpty()) {
|
||||||
logger.debug("Accepted MTB File for processing")
|
logger.debug("Accepted MTB File for processing")
|
||||||
if (requestId.isBlank()) {
|
if (requestId.isBlank()) {
|
||||||
requestProcessor.processMtbFile(mtbFile)
|
requestProcessor.processMtbFile(mtbFile)
|
||||||
@@ -88,9 +92,4 @@ class KafkaInputListener(
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
private fun handleDnpmV2Message(record: ConsumerRecord<String, String>) {
|
|
||||||
// Do not handle DNPM-V2 for now
|
|
||||||
logger.warn("Ignoring MTB File in DNPM V2 format: Not implemented yet")
|
|
||||||
}
|
|
||||||
|
|
||||||
}
|
}
|
||||||
|
@@ -19,8 +19,6 @@
|
|||||||
|
|
||||||
package dev.dnpm.etl.processor.input
|
package dev.dnpm.etl.processor.input
|
||||||
|
|
||||||
import de.ukw.ccc.bwhc.dto.Consent
|
|
||||||
import de.ukw.ccc.bwhc.dto.MtbFile
|
|
||||||
import dev.dnpm.etl.processor.CustomMediaType
|
import dev.dnpm.etl.processor.CustomMediaType
|
||||||
import dev.dnpm.etl.processor.PatientId
|
import dev.dnpm.etl.processor.PatientId
|
||||||
import dev.dnpm.etl.processor.consent.IGetConsent
|
import dev.dnpm.etl.processor.consent.IGetConsent
|
||||||
@@ -46,23 +44,7 @@ class MtbFileRestController(
|
|||||||
return ResponseEntity.ok("Test")
|
return ResponseEntity.ok("Test")
|
||||||
}
|
}
|
||||||
|
|
||||||
@PostMapping(consumes = [MediaType.APPLICATION_JSON_VALUE])
|
@PostMapping(consumes = [MediaType.APPLICATION_JSON_VALUE, CustomMediaType.APPLICATION_VND_DNPM_V2_MTB_JSON_VALUE])
|
||||||
fun mtbFile(@RequestBody mtbFile: MtbFile): ResponseEntity<Unit> {
|
|
||||||
val consentStatusBooleanPair = checkConsentStatus(mtbFile)
|
|
||||||
val ttpConsentStatus = consentStatusBooleanPair.first
|
|
||||||
val isConsentOK = consentStatusBooleanPair.second
|
|
||||||
if (isConsentOK) {
|
|
||||||
logger.debug("Accepted MTB File (bwHC V1) for processing")
|
|
||||||
requestProcessor.processMtbFile(mtbFile)
|
|
||||||
} else {
|
|
||||||
logger.debug("Accepted MTB File (bwHC V1) and process deletion")
|
|
||||||
val patientId = PatientId(mtbFile.patient.id)
|
|
||||||
requestProcessor.processDeletion(patientId, ttpConsentStatus)
|
|
||||||
}
|
|
||||||
return ResponseEntity.accepted().build()
|
|
||||||
}
|
|
||||||
|
|
||||||
@PostMapping(consumes = [CustomMediaType.APPLICATION_VND_DNPM_V2_MTB_JSON_VALUE])
|
|
||||||
fun mtbFile(@RequestBody mtbFile: Mtb): ResponseEntity<Unit> {
|
fun mtbFile(@RequestBody mtbFile: Mtb): ResponseEntity<Unit> {
|
||||||
logger.debug("Accepted MTB File (DNPM V2) for processing")
|
logger.debug("Accepted MTB File (DNPM V2) for processing")
|
||||||
requestProcessor.processMtbFile(mtbFile)
|
requestProcessor.processMtbFile(mtbFile)
|
||||||
@@ -76,17 +58,4 @@ class MtbFileRestController(
|
|||||||
return ResponseEntity.accepted().build()
|
return ResponseEntity.accepted().build()
|
||||||
}
|
}
|
||||||
|
|
||||||
private fun checkConsentStatus(mtbFile: MtbFile): Pair<TtpConsentStatus, Boolean> {
|
|
||||||
var ttpConsentStatus = iGetConsent.getTtpBroadConsentStatus(mtbFile.patient.id)
|
|
||||||
|
|
||||||
val isConsentOK = (ttpConsentStatus == TtpConsentStatus.UNKNOWN_CHECK_FILE && mtbFile.consent.status == Consent.Status.ACTIVE)
|
|
||||||
|| ttpConsentStatus == TtpConsentStatus.BROAD_CONSENT_GIVEN
|
|
||||||
|
|
||||||
if (ttpConsentStatus == TtpConsentStatus.UNKNOWN_CHECK_FILE && mtbFile.consent.status == Consent.Status.REJECTED) {
|
|
||||||
// in case ttp check is disabled - we propagate rejected status anyway
|
|
||||||
ttpConsentStatus = TtpConsentStatus.BROAD_CONSENT_MISSING_OR_REJECTED
|
|
||||||
}
|
|
||||||
return Pair(ttpConsentStatus, isConsentOK)
|
|
||||||
}
|
|
||||||
|
|
||||||
}
|
}
|
||||||
|
@@ -128,15 +128,11 @@ class RestConnectionCheckService(
|
|||||||
fun check() {
|
fun check() {
|
||||||
result = try {
|
result = try {
|
||||||
val available = restTemplate.getForEntity(
|
val available = restTemplate.getForEntity(
|
||||||
if (restTargetProperties.isBwhc) {
|
UriComponentsBuilder.fromUriString(restTargetProperties.uri.toString())
|
||||||
UriComponentsBuilder.fromUriString(restTargetProperties.uri.toString()).path("").toUriString()
|
.pathSegment("mtb")
|
||||||
} else {
|
.pathSegment("kaplan-meier")
|
||||||
UriComponentsBuilder.fromUriString(restTargetProperties.uri.toString())
|
.pathSegment("config")
|
||||||
.pathSegment("mtb")
|
.toUriString(),
|
||||||
.pathSegment("kaplan-meier")
|
|
||||||
.pathSegment("config")
|
|
||||||
.toUriString()
|
|
||||||
},
|
|
||||||
String::class.java
|
String::class.java
|
||||||
).statusCode == HttpStatus.OK
|
).statusCode == HttpStatus.OK
|
||||||
|
|
||||||
@@ -267,4 +263,4 @@ class GIcsConnectionCheckService(
|
|||||||
override fun connectionAvailable(): ConnectionCheckResult.GIcsConnectionCheckResult {
|
override fun connectionAvailable(): ConnectionCheckResult.GIcsConnectionCheckResult {
|
||||||
return this.result
|
return this.result
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
@@ -20,11 +20,11 @@
|
|||||||
package dev.dnpm.etl.processor.output
|
package dev.dnpm.etl.processor.output
|
||||||
|
|
||||||
import com.fasterxml.jackson.databind.ObjectMapper
|
import com.fasterxml.jackson.databind.ObjectMapper
|
||||||
import de.ukw.ccc.bwhc.dto.Consent
|
|
||||||
import de.ukw.ccc.bwhc.dto.MtbFile
|
|
||||||
import dev.dnpm.etl.processor.CustomMediaType
|
import dev.dnpm.etl.processor.CustomMediaType
|
||||||
import dev.dnpm.etl.processor.config.KafkaProperties
|
import dev.dnpm.etl.processor.config.KafkaProperties
|
||||||
import dev.dnpm.etl.processor.monitoring.RequestStatus
|
import dev.dnpm.etl.processor.monitoring.RequestStatus
|
||||||
|
import dev.pcvolkmer.mv64e.mtb.Mtb
|
||||||
|
import dev.pcvolkmer.mv64e.mtb.MvhMetadata
|
||||||
import org.apache.kafka.clients.producer.ProducerRecord
|
import org.apache.kafka.clients.producer.ProducerRecord
|
||||||
import org.slf4j.LoggerFactory
|
import org.slf4j.LoggerFactory
|
||||||
import org.springframework.http.MediaType
|
import org.springframework.http.MediaType
|
||||||
@@ -50,9 +50,6 @@ class KafkaMtbFileSender(
|
|||||||
objectMapper.writeValueAsString(request)
|
objectMapper.writeValueAsString(request)
|
||||||
)
|
)
|
||||||
when (request) {
|
when (request) {
|
||||||
is BwhcV1MtbFileRequest -> record.headers()
|
|
||||||
.add("contentType", MediaType.APPLICATION_JSON_VALUE.toByteArray())
|
|
||||||
|
|
||||||
is DnpmV2MtbFileRequest -> record.headers()
|
is DnpmV2MtbFileRequest -> record.headers()
|
||||||
.add(
|
.add(
|
||||||
"contentType",
|
"contentType",
|
||||||
@@ -75,13 +72,8 @@ class KafkaMtbFileSender(
|
|||||||
}
|
}
|
||||||
|
|
||||||
override fun send(request: DeleteRequest): MtbFileSender.Response {
|
override fun send(request: DeleteRequest): MtbFileSender.Response {
|
||||||
val dummyMtbFile = MtbFile.builder()
|
val dummyMtbFile = Mtb.builder()
|
||||||
.withConsent(
|
.metadata(MvhMetadata())
|
||||||
Consent.builder()
|
|
||||||
.withPatient(request.patientId.value)
|
|
||||||
.withStatus(Consent.Status.REJECTED)
|
|
||||||
.build()
|
|
||||||
)
|
|
||||||
.build()
|
.build()
|
||||||
|
|
||||||
return try {
|
return try {
|
||||||
@@ -92,7 +84,7 @@ class KafkaMtbFileSender(
|
|||||||
key(request),
|
key(request),
|
||||||
// Always use old BwhcV1FileRequest with Consent REJECT
|
// Always use old BwhcV1FileRequest with Consent REJECT
|
||||||
objectMapper.writeValueAsString(
|
objectMapper.writeValueAsString(
|
||||||
BwhcV1MtbFileRequest(
|
DnpmV2MtbFileRequest(
|
||||||
request.requestId,
|
request.requestId,
|
||||||
dummyMtbFile
|
dummyMtbFile
|
||||||
)
|
)
|
||||||
@@ -119,7 +111,6 @@ class KafkaMtbFileSender(
|
|||||||
|
|
||||||
private fun key(request: MtbRequest): String {
|
private fun key(request: MtbRequest): String {
|
||||||
return when (request) {
|
return when (request) {
|
||||||
is BwhcV1MtbFileRequest -> "{\"pid\": \"${request.content.patient.id}\"}"
|
|
||||||
is DnpmV2MtbFileRequest -> "{\"pid\": \"${request.content.patient.id}\"}"
|
is DnpmV2MtbFileRequest -> "{\"pid\": \"${request.content.patient.id}\"}"
|
||||||
is DeleteRequest -> "{\"pid\": \"${request.patientId.value}\"}"
|
is DeleteRequest -> "{\"pid\": \"${request.patientId.value}\"}"
|
||||||
else -> throw IllegalArgumentException("Unsupported request type: ${request::class.simpleName}")
|
else -> throw IllegalArgumentException("Unsupported request type: ${request::class.simpleName}")
|
||||||
|
@@ -19,7 +19,6 @@
|
|||||||
|
|
||||||
package dev.dnpm.etl.processor.output
|
package dev.dnpm.etl.processor.output
|
||||||
|
|
||||||
import de.ukw.ccc.bwhc.dto.MtbFile
|
|
||||||
import dev.dnpm.etl.processor.PatientPseudonym
|
import dev.dnpm.etl.processor.PatientPseudonym
|
||||||
import dev.dnpm.etl.processor.RequestId
|
import dev.dnpm.etl.processor.RequestId
|
||||||
import dev.pcvolkmer.mv64e.mtb.Mtb
|
import dev.pcvolkmer.mv64e.mtb.Mtb
|
||||||
@@ -35,15 +34,6 @@ sealed interface MtbFileRequest<out T> : MtbRequest {
|
|||||||
fun patientPseudonym(): PatientPseudonym
|
fun patientPseudonym(): PatientPseudonym
|
||||||
}
|
}
|
||||||
|
|
||||||
data class BwhcV1MtbFileRequest(
|
|
||||||
override val requestId: RequestId,
|
|
||||||
override val content: MtbFile
|
|
||||||
) : MtbFileRequest<MtbFile> {
|
|
||||||
override fun patientPseudonym(): PatientPseudonym {
|
|
||||||
return PatientPseudonym(content.patient.id)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
data class DnpmV2MtbFileRequest(
|
data class DnpmV2MtbFileRequest(
|
||||||
override val requestId: RequestId,
|
override val requestId: RequestId,
|
||||||
override val content: Mtb
|
override val content: Mtb
|
||||||
|
@@ -1,51 +0,0 @@
|
|||||||
/*
|
|
||||||
* This file is part of ETL-Processor
|
|
||||||
*
|
|
||||||
* Copyright (c) 2025 Comprehensive Cancer Center Mainfranken, Datenintegrationszentrum Philipps-Universität Marburg and Contributors
|
|
||||||
*
|
|
||||||
* This program is free software: you can redistribute it and/or modify
|
|
||||||
* it under the terms of the GNU Affero General Public License as published
|
|
||||||
* by the Free Software Foundation, either version 3 of the License, or
|
|
||||||
* (at your option) any later version.
|
|
||||||
*
|
|
||||||
* This program is distributed in the hope that it will be useful,
|
|
||||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
|
||||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
|
||||||
* GNU Affero General Public License for more details.
|
|
||||||
*
|
|
||||||
* You should have received a copy of the GNU Affero General Public License
|
|
||||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
|
||||||
*/
|
|
||||||
|
|
||||||
package dev.dnpm.etl.processor.output
|
|
||||||
|
|
||||||
import dev.dnpm.etl.processor.PatientPseudonym
|
|
||||||
import dev.dnpm.etl.processor.config.RestTargetProperties
|
|
||||||
import dev.dnpm.etl.processor.monitoring.ReportService
|
|
||||||
import org.springframework.retry.support.RetryTemplate
|
|
||||||
import org.springframework.web.client.RestTemplate
|
|
||||||
import org.springframework.web.util.UriComponentsBuilder
|
|
||||||
|
|
||||||
class RestBwhcMtbFileSender(
|
|
||||||
restTemplate: RestTemplate,
|
|
||||||
private val restTargetProperties: RestTargetProperties,
|
|
||||||
retryTemplate: RetryTemplate,
|
|
||||||
reportService: ReportService,
|
|
||||||
) : RestMtbFileSender(restTemplate, restTargetProperties, retryTemplate, reportService) {
|
|
||||||
|
|
||||||
override fun sendUrl(): String {
|
|
||||||
return UriComponentsBuilder
|
|
||||||
.fromUriString(restTargetProperties.uri.toString())
|
|
||||||
.pathSegment("MTBFile")
|
|
||||||
.toUriString()
|
|
||||||
}
|
|
||||||
|
|
||||||
override fun deleteUrl(patientId: PatientPseudonym): String {
|
|
||||||
return UriComponentsBuilder
|
|
||||||
.fromUriString(restTargetProperties.uri.toString())
|
|
||||||
.pathSegment("Patient")
|
|
||||||
.pathSegment(patientId.value)
|
|
||||||
.toUriString()
|
|
||||||
}
|
|
||||||
|
|
||||||
}
|
|
@@ -108,7 +108,6 @@ abstract class RestMtbFileSender(
|
|||||||
val password = restTargetProperties.password
|
val password = restTargetProperties.password
|
||||||
val headers = HttpHeaders()
|
val headers = HttpHeaders()
|
||||||
headers.contentType = when (request) {
|
headers.contentType = when (request) {
|
||||||
is BwhcV1MtbFileRequest -> MediaType.APPLICATION_JSON
|
|
||||||
is DnpmV2MtbFileRequest -> CustomMediaType.APPLICATION_VND_DNPM_V2_MTB_JSON
|
is DnpmV2MtbFileRequest -> CustomMediaType.APPLICATION_VND_DNPM_V2_MTB_JSON
|
||||||
else -> MediaType.APPLICATION_JSON
|
else -> MediaType.APPLICATION_JSON
|
||||||
}
|
}
|
||||||
|
@@ -19,217 +19,12 @@
|
|||||||
|
|
||||||
package dev.dnpm.etl.processor.pseudonym
|
package dev.dnpm.etl.processor.pseudonym
|
||||||
|
|
||||||
import de.ukw.ccc.bwhc.dto.MtbFile
|
|
||||||
import dev.dnpm.etl.processor.PatientId
|
import dev.dnpm.etl.processor.PatientId
|
||||||
import dev.pcvolkmer.mv64e.mtb.ModelProjectConsent
|
import dev.pcvolkmer.mv64e.mtb.ModelProjectConsent
|
||||||
import dev.pcvolkmer.mv64e.mtb.Mtb
|
import dev.pcvolkmer.mv64e.mtb.Mtb
|
||||||
import dev.pcvolkmer.mv64e.mtb.MvhMetadata
|
import dev.pcvolkmer.mv64e.mtb.MvhMetadata
|
||||||
import org.apache.commons.codec.digest.DigestUtils
|
import org.apache.commons.codec.digest.DigestUtils
|
||||||
|
|
||||||
/** Replaces patient ID with generated patient pseudonym
|
|
||||||
*
|
|
||||||
* @param pseudonymizeService The pseudonymizeService to be used
|
|
||||||
* @return The MTB file containing patient pseudonymes
|
|
||||||
*/
|
|
||||||
infix fun MtbFile.pseudonymizeWith(pseudonymizeService: PseudonymizeService) {
|
|
||||||
val patientPseudonym = pseudonymizeService.patientPseudonym(PatientId(this.patient.id)).value
|
|
||||||
|
|
||||||
this.episode?.patient = patientPseudonym
|
|
||||||
this.carePlans?.forEach { it.patient = patientPseudonym }
|
|
||||||
this.patient.id = patientPseudonym
|
|
||||||
this.claims?.forEach { it.patient = patientPseudonym }
|
|
||||||
this.consent?.patient = patientPseudonym
|
|
||||||
this.claimResponses?.forEach { it.patient = patientPseudonym }
|
|
||||||
this.diagnoses?.forEach { it.patient = patientPseudonym }
|
|
||||||
this.ecogStatus?.forEach { it.patient = patientPseudonym }
|
|
||||||
this.familyMemberDiagnoses?.forEach { it.patient = patientPseudonym }
|
|
||||||
this.geneticCounsellingRequests?.forEach { it.patient = patientPseudonym }
|
|
||||||
this.histologyReevaluationRequests?.forEach { it.patient = patientPseudonym }
|
|
||||||
this.histologyReports?.forEach {
|
|
||||||
it.patient = patientPseudonym
|
|
||||||
it.tumorMorphology?.patient = patientPseudonym
|
|
||||||
}
|
|
||||||
this.lastGuidelineTherapies?.forEach { it.patient = patientPseudonym }
|
|
||||||
this.molecularPathologyFindings?.forEach { it.patient = patientPseudonym }
|
|
||||||
this.molecularTherapies?.forEach { molecularTherapy ->
|
|
||||||
molecularTherapy.history.forEach {
|
|
||||||
it.patient = patientPseudonym
|
|
||||||
}
|
|
||||||
}
|
|
||||||
this.ngsReports?.forEach { it.patient = patientPseudonym }
|
|
||||||
this.previousGuidelineTherapies?.forEach { it.patient = patientPseudonym }
|
|
||||||
this.rebiopsyRequests?.forEach { it.patient = patientPseudonym }
|
|
||||||
this.recommendations?.forEach { it.patient = patientPseudonym }
|
|
||||||
this.responses?.forEach { it.patient = patientPseudonym }
|
|
||||||
this.studyInclusionRequests?.forEach { it.patient = patientPseudonym }
|
|
||||||
this.specimens?.forEach { it.patient = patientPseudonym }
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Creates new hash of content IDs with given prefix except for patient IDs
|
|
||||||
*
|
|
||||||
* @param pseudonymizeService The pseudonymizeService to be used
|
|
||||||
* @return The MTB file containing rehashed content IDs
|
|
||||||
*/
|
|
||||||
infix fun MtbFile.anonymizeContentWith(pseudonymizeService: PseudonymizeService) {
|
|
||||||
val prefix = pseudonymizeService.prefix()
|
|
||||||
|
|
||||||
fun anonymize(id: String): String {
|
|
||||||
val hash = DigestUtils.sha256Hex("$prefix-$id").substring(0, 41).lowercase()
|
|
||||||
return "$prefix$hash"
|
|
||||||
}
|
|
||||||
|
|
||||||
this.episode?.apply {
|
|
||||||
id = id?.let {
|
|
||||||
anonymize(it)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
this.carePlans?.onEach { carePlan ->
|
|
||||||
carePlan?.apply {
|
|
||||||
id = id?.let { anonymize(it) }
|
|
||||||
diagnosis = diagnosis?.let { anonymize(it) }
|
|
||||||
geneticCounsellingRequest = geneticCounsellingRequest?.let { anonymize(it) }
|
|
||||||
rebiopsyRequests = rebiopsyRequests.map { it?.let { anonymize(it) } }
|
|
||||||
recommendations = recommendations.map { it?.let { anonymize(it) } }
|
|
||||||
studyInclusionRequests = studyInclusionRequests.map { it?.let { anonymize(it) } }
|
|
||||||
}
|
|
||||||
}
|
|
||||||
this.claims?.onEach { claim ->
|
|
||||||
claim?.apply {
|
|
||||||
id = id?.let { anonymize(it) }
|
|
||||||
therapy = therapy?.let { anonymize(it) }
|
|
||||||
}
|
|
||||||
}
|
|
||||||
this.claimResponses?.onEach { claimResponse ->
|
|
||||||
claimResponse?.apply {
|
|
||||||
id = id?.let { anonymize(it) }
|
|
||||||
claim = claim?.let { anonymize(it) }
|
|
||||||
}
|
|
||||||
}
|
|
||||||
this.consent?.apply {
|
|
||||||
id = id?.let { anonymize(it) }
|
|
||||||
}
|
|
||||||
this.diagnoses?.onEach { diagnosis ->
|
|
||||||
diagnosis?.apply {
|
|
||||||
id = id?.let { anonymize(it) }
|
|
||||||
histologyResults = histologyResults?.map { it?.let { anonymize(it) } }
|
|
||||||
}
|
|
||||||
}
|
|
||||||
this.ecogStatus?.onEach { ecogStatus ->
|
|
||||||
ecogStatus?.apply {
|
|
||||||
id = id?.let { anonymize(it) }
|
|
||||||
}
|
|
||||||
}
|
|
||||||
this.familyMemberDiagnoses?.onEach { familyMemberDiagnosis ->
|
|
||||||
familyMemberDiagnosis?.apply {
|
|
||||||
id = id?.let { anonymize(it) }
|
|
||||||
}
|
|
||||||
}
|
|
||||||
this.geneticCounsellingRequests?.onEach { geneticCounsellingRequest ->
|
|
||||||
geneticCounsellingRequest?.apply {
|
|
||||||
id = id?.let { anonymize(it) }
|
|
||||||
}
|
|
||||||
}
|
|
||||||
this.histologyReevaluationRequests?.onEach { histologyReevaluationRequest ->
|
|
||||||
histologyReevaluationRequest?.apply {
|
|
||||||
id = id?.let { anonymize(it) }
|
|
||||||
specimen = specimen?.let { anonymize(it) }
|
|
||||||
}
|
|
||||||
}
|
|
||||||
this.histologyReports?.onEach { histologyReport ->
|
|
||||||
histologyReport?.apply {
|
|
||||||
id = id?.let { anonymize(it) }
|
|
||||||
specimen = specimen?.let { anonymize(it) }
|
|
||||||
tumorMorphology?.apply {
|
|
||||||
id = id?.let { anonymize(it) }
|
|
||||||
specimen = specimen?.let { anonymize(it) }
|
|
||||||
}
|
|
||||||
tumorCellContent?.apply {
|
|
||||||
id = id?.let { anonymize(it) }
|
|
||||||
specimen = specimen?.let { anonymize(it) }
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
this.lastGuidelineTherapies?.onEach { lastGuidelineTherapy ->
|
|
||||||
lastGuidelineTherapy?.apply {
|
|
||||||
id = id?.let { anonymize(it) }
|
|
||||||
diagnosis = diagnosis?.let { anonymize(it) }
|
|
||||||
}
|
|
||||||
}
|
|
||||||
this.molecularPathologyFindings?.onEach { molecularPathologyFinding ->
|
|
||||||
molecularPathologyFinding?.apply {
|
|
||||||
id = id?.let { anonymize(it) }
|
|
||||||
specimen = specimen?.let { anonymize(it) }
|
|
||||||
}
|
|
||||||
}
|
|
||||||
this.molecularTherapies?.onEach { molecularTherapy ->
|
|
||||||
molecularTherapy?.apply {
|
|
||||||
history?.onEach { history ->
|
|
||||||
history?.apply {
|
|
||||||
id = id?.let { anonymize(it) }
|
|
||||||
basedOn = basedOn?.let { anonymize(it) }
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
this.ngsReports?.onEach { ngsReport ->
|
|
||||||
ngsReport?.apply {
|
|
||||||
id = id?.let { anonymize(it) }
|
|
||||||
specimen = specimen?.let { anonymize(it) }
|
|
||||||
tumorCellContent?.apply {
|
|
||||||
id = id?.let { anonymize(it) }
|
|
||||||
specimen = specimen?.let { anonymize(it) }
|
|
||||||
}
|
|
||||||
simpleVariants?.onEach { simpleVariant ->
|
|
||||||
simpleVariant?.apply {
|
|
||||||
id = id?.let { anonymize(it) }
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
this.previousGuidelineTherapies?.onEach { previousGuidelineTherapy ->
|
|
||||||
previousGuidelineTherapy?.apply {
|
|
||||||
id = id?.let { anonymize(it) }
|
|
||||||
diagnosis = diagnosis?.let { anonymize(it) }
|
|
||||||
medication.forEach { medication ->
|
|
||||||
medication?.apply {
|
|
||||||
id = id?.let { anonymize(it) }
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
this.rebiopsyRequests?.onEach { rebiopsyRequest ->
|
|
||||||
rebiopsyRequest?.apply {
|
|
||||||
id = id?.let { anonymize(it) }
|
|
||||||
specimen = specimen?.let { anonymize(it) }
|
|
||||||
}
|
|
||||||
}
|
|
||||||
this.recommendations?.onEach { recommendation ->
|
|
||||||
recommendation?.apply {
|
|
||||||
id = id?.let { anonymize(it) }
|
|
||||||
diagnosis = diagnosis?.let { anonymize(it) }
|
|
||||||
ngsReport = ngsReport?.let { anonymize(it) }
|
|
||||||
}
|
|
||||||
}
|
|
||||||
this.responses?.onEach { response ->
|
|
||||||
response?.apply {
|
|
||||||
id = id?.let { anonymize(it) }
|
|
||||||
therapy = therapy?.let { anonymize(it) }
|
|
||||||
}
|
|
||||||
}
|
|
||||||
this.studyInclusionRequests?.onEach { studyInclusionRequest ->
|
|
||||||
studyInclusionRequest?.apply {
|
|
||||||
id = id?.let { anonymize(it) }
|
|
||||||
reason = reason?.let { anonymize(it) }
|
|
||||||
}
|
|
||||||
}
|
|
||||||
this.specimens?.onEach { specimen ->
|
|
||||||
specimen?.apply {
|
|
||||||
id = id?.let { anonymize(it) }
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
/** Replaces patient ID with generated patient pseudonym
|
/** Replaces patient ID with generated patient pseudonym
|
||||||
*
|
*
|
||||||
* @since 0.11.0
|
* @since 0.11.0
|
||||||
@@ -353,4 +148,4 @@ fun Mtb.ensureMetaDataIsInitialized() {
|
|||||||
infix fun Mtb.addGenomDeTan(pseudonymizeService: PseudonymizeService)
|
infix fun Mtb.addGenomDeTan(pseudonymizeService: PseudonymizeService)
|
||||||
{
|
{
|
||||||
this.metadata.transferTan = pseudonymizeService.genomDeTan(PatientId(this.patient.id))
|
this.metadata.transferTan = pseudonymizeService.genomDeTan(PatientId(this.patient.id))
|
||||||
}
|
}
|
||||||
|
@@ -20,7 +20,6 @@
|
|||||||
package dev.dnpm.etl.processor.services
|
package dev.dnpm.etl.processor.services
|
||||||
|
|
||||||
import com.fasterxml.jackson.databind.ObjectMapper
|
import com.fasterxml.jackson.databind.ObjectMapper
|
||||||
import de.ukw.ccc.bwhc.dto.MtbFile
|
|
||||||
import dev.dnpm.etl.processor.*
|
import dev.dnpm.etl.processor.*
|
||||||
import dev.dnpm.etl.processor.config.AppConfigProperties
|
import dev.dnpm.etl.processor.config.AppConfigProperties
|
||||||
import dev.dnpm.etl.processor.consent.TtpConsentStatus
|
import dev.dnpm.etl.processor.consent.TtpConsentStatus
|
||||||
@@ -28,7 +27,10 @@ import dev.dnpm.etl.processor.monitoring.Report
|
|||||||
import dev.dnpm.etl.processor.monitoring.Request
|
import dev.dnpm.etl.processor.monitoring.Request
|
||||||
import dev.dnpm.etl.processor.monitoring.RequestStatus
|
import dev.dnpm.etl.processor.monitoring.RequestStatus
|
||||||
import dev.dnpm.etl.processor.monitoring.RequestType
|
import dev.dnpm.etl.processor.monitoring.RequestType
|
||||||
import dev.dnpm.etl.processor.output.*
|
import dev.dnpm.etl.processor.output.DeleteRequest
|
||||||
|
import dev.dnpm.etl.processor.output.DnpmV2MtbFileRequest
|
||||||
|
import dev.dnpm.etl.processor.output.MtbFileRequest
|
||||||
|
import dev.dnpm.etl.processor.output.MtbFileSender
|
||||||
import dev.dnpm.etl.processor.pseudonym.PseudonymizeService
|
import dev.dnpm.etl.processor.pseudonym.PseudonymizeService
|
||||||
import dev.dnpm.etl.processor.pseudonym.addGenomDeTan
|
import dev.dnpm.etl.processor.pseudonym.addGenomDeTan
|
||||||
import dev.dnpm.etl.processor.pseudonym.anonymizeContentWith
|
import dev.dnpm.etl.processor.pseudonym.anonymizeContentWith
|
||||||
@@ -42,7 +44,6 @@ import org.slf4j.Logger
|
|||||||
import org.slf4j.LoggerFactory
|
import org.slf4j.LoggerFactory
|
||||||
import org.springframework.context.ApplicationEventPublisher
|
import org.springframework.context.ApplicationEventPublisher
|
||||||
import org.springframework.stereotype.Service
|
import org.springframework.stereotype.Service
|
||||||
import java.lang.RuntimeException
|
|
||||||
import java.time.Instant
|
import java.time.Instant
|
||||||
import java.util.*
|
import java.util.*
|
||||||
|
|
||||||
@@ -59,17 +60,6 @@ class RequestProcessor(
|
|||||||
) {
|
) {
|
||||||
|
|
||||||
private var logger: Logger = LoggerFactory.getLogger("RequestProcessor")
|
private var logger: Logger = LoggerFactory.getLogger("RequestProcessor")
|
||||||
fun processMtbFile(mtbFile: MtbFile) {
|
|
||||||
processMtbFile(mtbFile, randomRequestId())
|
|
||||||
}
|
|
||||||
|
|
||||||
fun processMtbFile(mtbFile: MtbFile, requestId: RequestId) {
|
|
||||||
val pid = PatientId(mtbFile.patient.id)
|
|
||||||
mtbFile pseudonymizeWith pseudonymizeService
|
|
||||||
mtbFile anonymizeContentWith pseudonymizeService
|
|
||||||
val request = BwhcV1MtbFileRequest(requestId, transformationService.transform(mtbFile))
|
|
||||||
saveAndSend(request, pid)
|
|
||||||
}
|
|
||||||
|
|
||||||
fun processMtbFile(mtbFile: Mtb) {
|
fun processMtbFile(mtbFile: Mtb) {
|
||||||
processMtbFile(mtbFile, randomRequestId())
|
processMtbFile(mtbFile, randomRequestId())
|
||||||
@@ -144,7 +134,6 @@ class RequestProcessor(
|
|||||||
|
|
||||||
private fun <T> isDuplication(pseudonymizedMtbFileRequest: MtbFileRequest<T>): Boolean {
|
private fun <T> isDuplication(pseudonymizedMtbFileRequest: MtbFileRequest<T>): Boolean {
|
||||||
val patientPseudonym = when (pseudonymizedMtbFileRequest) {
|
val patientPseudonym = when (pseudonymizedMtbFileRequest) {
|
||||||
is BwhcV1MtbFileRequest -> PatientPseudonym(pseudonymizedMtbFileRequest.content.patient.id)
|
|
||||||
is DnpmV2MtbFileRequest -> PatientPseudonym(pseudonymizedMtbFileRequest.content.patient.id)
|
is DnpmV2MtbFileRequest -> PatientPseudonym(pseudonymizedMtbFileRequest.content.patient.id)
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -214,7 +203,6 @@ class RequestProcessor(
|
|||||||
|
|
||||||
private fun <T> fingerprint(request: MtbFileRequest<T>): Fingerprint {
|
private fun <T> fingerprint(request: MtbFileRequest<T>): Fingerprint {
|
||||||
return when (request) {
|
return when (request) {
|
||||||
is BwhcV1MtbFileRequest -> fingerprint(objectMapper.writeValueAsString(request.content))
|
|
||||||
is DnpmV2MtbFileRequest -> fingerprint(objectMapper.writeValueAsString(request.content))
|
is DnpmV2MtbFileRequest -> fingerprint(objectMapper.writeValueAsString(request.content))
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
@@ -22,14 +22,9 @@ package dev.dnpm.etl.processor.services
|
|||||||
import com.fasterxml.jackson.databind.ObjectMapper
|
import com.fasterxml.jackson.databind.ObjectMapper
|
||||||
import com.jayway.jsonpath.JsonPath
|
import com.jayway.jsonpath.JsonPath
|
||||||
import com.jayway.jsonpath.PathNotFoundException
|
import com.jayway.jsonpath.PathNotFoundException
|
||||||
import de.ukw.ccc.bwhc.dto.MtbFile
|
|
||||||
import dev.pcvolkmer.mv64e.mtb.Mtb
|
import dev.pcvolkmer.mv64e.mtb.Mtb
|
||||||
|
|
||||||
class TransformationService(private val objectMapper: ObjectMapper, private val transformations: List<Transformation>) {
|
class TransformationService(private val objectMapper: ObjectMapper, private val transformations: List<Transformation>) {
|
||||||
fun transform(mtbFile: MtbFile): MtbFile {
|
|
||||||
val json = transform(objectMapper.writeValueAsString(mtbFile))
|
|
||||||
return objectMapper.readValue(json, MtbFile::class.java)
|
|
||||||
}
|
|
||||||
|
|
||||||
fun transform(mtbFile: Mtb): Mtb {
|
fun transform(mtbFile: Mtb): Mtb {
|
||||||
val json = transform(objectMapper.writeValueAsString(mtbFile))
|
val json = transform(objectMapper.writeValueAsString(mtbFile))
|
||||||
|
@@ -20,7 +20,7 @@ spring:
|
|||||||
|
|
||||||
app:
|
app:
|
||||||
rest:
|
rest:
|
||||||
uri: http://localhost:9000/bwhc/etl/api
|
uri: http://localhost/api
|
||||||
#kafka:
|
#kafka:
|
||||||
# topic: test
|
# topic: test
|
||||||
# response-topic: test_response
|
# response-topic: test_response
|
||||||
|
@@ -20,12 +20,10 @@
|
|||||||
package dev.dnpm.etl.processor.input
|
package dev.dnpm.etl.processor.input
|
||||||
|
|
||||||
import com.fasterxml.jackson.databind.ObjectMapper
|
import com.fasterxml.jackson.databind.ObjectMapper
|
||||||
import de.ukw.ccc.bwhc.dto.Consent
|
|
||||||
import de.ukw.ccc.bwhc.dto.MtbFile
|
|
||||||
import de.ukw.ccc.bwhc.dto.Patient
|
|
||||||
import dev.dnpm.etl.processor.consent.TtpConsentStatus
|
import dev.dnpm.etl.processor.consent.TtpConsentStatus
|
||||||
import dev.dnpm.etl.processor.CustomMediaType
|
import dev.dnpm.etl.processor.CustomMediaType
|
||||||
import dev.dnpm.etl.processor.services.RequestProcessor
|
import dev.dnpm.etl.processor.services.RequestProcessor
|
||||||
|
import dev.pcvolkmer.mv64e.mtb.*
|
||||||
import org.apache.kafka.clients.consumer.ConsumerRecord
|
import org.apache.kafka.clients.consumer.ConsumerRecord
|
||||||
import org.apache.kafka.common.header.internals.RecordHeader
|
import org.apache.kafka.common.header.internals.RecordHeader
|
||||||
import org.apache.kafka.common.header.internals.RecordHeaders
|
import org.apache.kafka.common.header.internals.RecordHeaders
|
||||||
@@ -57,9 +55,20 @@ class KafkaInputListenerTest {
|
|||||||
|
|
||||||
@Test
|
@Test
|
||||||
fun shouldProcessMtbFileRequest() {
|
fun shouldProcessMtbFileRequest() {
|
||||||
val mtbFile = MtbFile.builder()
|
val mtbFile = Mtb.builder()
|
||||||
.withPatient(Patient.builder().withId("DUMMY_12345678").build())
|
.patient(Patient.builder().id("DUMMY_12345678").build())
|
||||||
.withConsent(Consent.builder().withStatus(Consent.Status.ACTIVE).build())
|
.metadata(
|
||||||
|
MvhMetadata
|
||||||
|
.builder()
|
||||||
|
.modelProjectConsent(
|
||||||
|
ModelProjectConsent
|
||||||
|
.builder()
|
||||||
|
.provisions(
|
||||||
|
listOf(Provision.builder().type(ConsentProvision.PERMIT).purpose(ModelProjectConsentPurpose.SEQUENCING).build())
|
||||||
|
).build()
|
||||||
|
)
|
||||||
|
.build()
|
||||||
|
)
|
||||||
.build()
|
.build()
|
||||||
|
|
||||||
kafkaInputListener.onMessage(
|
kafkaInputListener.onMessage(
|
||||||
@@ -72,14 +81,25 @@ class KafkaInputListenerTest {
|
|||||||
)
|
)
|
||||||
)
|
)
|
||||||
|
|
||||||
verify(requestProcessor, times(1)).processMtbFile(any<MtbFile>())
|
verify(requestProcessor, times(1)).processMtbFile(any<Mtb>())
|
||||||
}
|
}
|
||||||
|
|
||||||
@Test
|
@Test
|
||||||
fun shouldProcessDeleteRequest() {
|
fun shouldProcessDeleteRequest() {
|
||||||
val mtbFile = MtbFile.builder()
|
val mtbFile = Mtb.builder()
|
||||||
.withPatient(Patient.builder().withId("DUMMY_12345678").build())
|
.patient(Patient.builder().id("DUMMY_12345678").build())
|
||||||
.withConsent(Consent.builder().withStatus(Consent.Status.REJECTED).build())
|
.metadata(
|
||||||
|
MvhMetadata
|
||||||
|
.builder()
|
||||||
|
.modelProjectConsent(
|
||||||
|
ModelProjectConsent
|
||||||
|
.builder()
|
||||||
|
.provisions(
|
||||||
|
listOf(Provision.builder().type(ConsentProvision.DENY).purpose(ModelProjectConsentPurpose.SEQUENCING).build())
|
||||||
|
).build()
|
||||||
|
)
|
||||||
|
.build()
|
||||||
|
)
|
||||||
.build()
|
.build()
|
||||||
|
|
||||||
kafkaInputListener.onMessage(
|
kafkaInputListener.onMessage(
|
||||||
@@ -100,9 +120,20 @@ class KafkaInputListenerTest {
|
|||||||
|
|
||||||
@Test
|
@Test
|
||||||
fun shouldProcessMtbFileRequestWithExistingRequestId() {
|
fun shouldProcessMtbFileRequestWithExistingRequestId() {
|
||||||
val mtbFile = MtbFile.builder()
|
val mtbFile = Mtb.builder()
|
||||||
.withPatient(Patient.builder().withId("DUMMY_12345678").build())
|
.patient(Patient.builder().id("DUMMY_12345678").build())
|
||||||
.withConsent(Consent.builder().withStatus(Consent.Status.ACTIVE).build())
|
.metadata(
|
||||||
|
MvhMetadata
|
||||||
|
.builder()
|
||||||
|
.modelProjectConsent(
|
||||||
|
ModelProjectConsent
|
||||||
|
.builder()
|
||||||
|
.provisions(
|
||||||
|
listOf(Provision.builder().type(ConsentProvision.PERMIT).purpose(ModelProjectConsentPurpose.SEQUENCING).build())
|
||||||
|
).build()
|
||||||
|
)
|
||||||
|
.build()
|
||||||
|
)
|
||||||
.build()
|
.build()
|
||||||
|
|
||||||
val headers = RecordHeaders(listOf(RecordHeader("requestId", UUID.randomUUID().toString().toByteArray())))
|
val headers = RecordHeaders(listOf(RecordHeader("requestId", UUID.randomUUID().toString().toByteArray())))
|
||||||
@@ -122,14 +153,25 @@ class KafkaInputListenerTest {
|
|||||||
)
|
)
|
||||||
)
|
)
|
||||||
|
|
||||||
verify(requestProcessor, times(1)).processMtbFile(any<MtbFile>(), anyValueClass())
|
verify(requestProcessor, times(1)).processMtbFile(any<Mtb>(), anyValueClass())
|
||||||
}
|
}
|
||||||
|
|
||||||
@Test
|
@Test
|
||||||
fun shouldProcessDeleteRequestWithExistingRequestId() {
|
fun shouldProcessDeleteRequestWithExistingRequestId() {
|
||||||
val mtbFile = MtbFile.builder()
|
val mtbFile = Mtb.builder()
|
||||||
.withPatient(Patient.builder().withId("DUMMY_12345678").build())
|
.patient(Patient.builder().id("DUMMY_12345678").build())
|
||||||
.withConsent(Consent.builder().withStatus(Consent.Status.REJECTED).build())
|
.metadata(
|
||||||
|
MvhMetadata
|
||||||
|
.builder()
|
||||||
|
.modelProjectConsent(
|
||||||
|
ModelProjectConsent
|
||||||
|
.builder()
|
||||||
|
.provisions(
|
||||||
|
listOf(Provision.builder().type(ConsentProvision.DENY).purpose(ModelProjectConsentPurpose.SEQUENCING).build())
|
||||||
|
).build()
|
||||||
|
)
|
||||||
|
.build()
|
||||||
|
)
|
||||||
.build()
|
.build()
|
||||||
|
|
||||||
val headers = RecordHeaders(listOf(RecordHeader("requestId", UUID.randomUUID().toString().toByteArray())))
|
val headers = RecordHeaders(listOf(RecordHeader("requestId", UUID.randomUUID().toString().toByteArray())))
|
||||||
@@ -148,15 +190,29 @@ class KafkaInputListenerTest {
|
|||||||
Optional.empty()
|
Optional.empty()
|
||||||
)
|
)
|
||||||
)
|
)
|
||||||
verify(requestProcessor, times(1)).processDeletion(anyValueClass(), anyValueClass(), eq(
|
verify(requestProcessor, times(1)).processDeletion(
|
||||||
TtpConsentStatus.UNKNOWN_CHECK_FILE))
|
anyValueClass(), anyValueClass(), eq(
|
||||||
|
TtpConsentStatus.UNKNOWN_CHECK_FILE
|
||||||
|
)
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
@Test
|
@Test
|
||||||
fun shouldNotProcessDnpmV2Request() {
|
fun shouldNotProcessDnpmV2Request() {
|
||||||
val mtbFile = MtbFile.builder()
|
val mtbFile = Mtb.builder()
|
||||||
.withPatient(Patient.builder().withId("DUMMY_12345678").build())
|
.patient(Patient.builder().id("DUMMY_12345678").build())
|
||||||
.withConsent(Consent.builder().withStatus(Consent.Status.REJECTED).build())
|
.metadata(
|
||||||
|
MvhMetadata
|
||||||
|
.builder()
|
||||||
|
.modelProjectConsent(
|
||||||
|
ModelProjectConsent
|
||||||
|
.builder()
|
||||||
|
.provisions(
|
||||||
|
listOf(Provision.builder().type(ConsentProvision.DENY).purpose(ModelProjectConsentPurpose.SEQUENCING).build())
|
||||||
|
).build()
|
||||||
|
)
|
||||||
|
.build()
|
||||||
|
)
|
||||||
.build()
|
.build()
|
||||||
|
|
||||||
val headers = RecordHeaders(
|
val headers = RecordHeaders(
|
||||||
@@ -180,8 +236,11 @@ class KafkaInputListenerTest {
|
|||||||
Optional.empty()
|
Optional.empty()
|
||||||
)
|
)
|
||||||
)
|
)
|
||||||
verify(requestProcessor, times(0)).processDeletion(anyValueClass(), anyValueClass(), eq(
|
verify(requestProcessor, times(0)).processDeletion(
|
||||||
TtpConsentStatus.UNKNOWN_CHECK_FILE))
|
anyValueClass(), anyValueClass(), eq(
|
||||||
|
TtpConsentStatus.UNKNOWN_CHECK_FILE
|
||||||
|
)
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
}
|
}
|
||||||
|
@@ -20,31 +20,21 @@
|
|||||||
package dev.dnpm.etl.processor.input
|
package dev.dnpm.etl.processor.input
|
||||||
|
|
||||||
import com.fasterxml.jackson.databind.ObjectMapper
|
import com.fasterxml.jackson.databind.ObjectMapper
|
||||||
import de.ukw.ccc.bwhc.dto.*
|
|
||||||
import de.ukw.ccc.bwhc.dto.Consent.Status
|
|
||||||
import dev.dnpm.etl.processor.CustomMediaType
|
import dev.dnpm.etl.processor.CustomMediaType
|
||||||
import dev.dnpm.etl.processor.consent.ConsentByMtbFile
|
|
||||||
import dev.dnpm.etl.processor.consent.GicsConsentService
|
import dev.dnpm.etl.processor.consent.GicsConsentService
|
||||||
import dev.dnpm.etl.processor.consent.TtpConsentStatus
|
|
||||||
import dev.dnpm.etl.processor.services.RequestProcessor
|
import dev.dnpm.etl.processor.services.RequestProcessor
|
||||||
import dev.pcvolkmer.mv64e.mtb.Mtb
|
import dev.pcvolkmer.mv64e.mtb.Mtb
|
||||||
import org.junit.jupiter.api.BeforeEach
|
import org.junit.jupiter.api.BeforeEach
|
||||||
import org.junit.jupiter.api.Nested
|
import org.junit.jupiter.api.Nested
|
||||||
import org.junit.jupiter.api.Test
|
import org.junit.jupiter.api.Test
|
||||||
import org.junit.jupiter.api.extension.ExtendWith
|
import org.junit.jupiter.api.extension.ExtendWith
|
||||||
import org.junit.jupiter.params.ParameterizedTest
|
|
||||||
import org.junit.jupiter.params.provider.ValueSource
|
|
||||||
import org.mockito.Mock
|
import org.mockito.Mock
|
||||||
import org.mockito.Mockito.times
|
import org.mockito.Mockito.times
|
||||||
import org.mockito.Mockito.verify
|
import org.mockito.Mockito.verify
|
||||||
import org.mockito.junit.jupiter.MockitoExtension
|
import org.mockito.junit.jupiter.MockitoExtension
|
||||||
import org.mockito.kotlin.any
|
import org.mockito.kotlin.any
|
||||||
import org.mockito.kotlin.anyValueClass
|
|
||||||
import org.mockito.kotlin.whenever
|
|
||||||
import org.springframework.core.io.ClassPathResource
|
import org.springframework.core.io.ClassPathResource
|
||||||
import org.springframework.http.MediaType
|
|
||||||
import org.springframework.test.web.servlet.MockMvc
|
import org.springframework.test.web.servlet.MockMvc
|
||||||
import org.springframework.test.web.servlet.delete
|
|
||||||
import org.springframework.test.web.servlet.post
|
import org.springframework.test.web.servlet.post
|
||||||
import org.springframework.test.web.servlet.setup.MockMvcBuilders
|
import org.springframework.test.web.servlet.setup.MockMvcBuilders
|
||||||
|
|
||||||
@@ -53,219 +43,6 @@ class MtbFileRestControllerTest {
|
|||||||
|
|
||||||
private val objectMapper = ObjectMapper()
|
private val objectMapper = ObjectMapper()
|
||||||
|
|
||||||
@Nested
|
|
||||||
inner class BwhcRequests {
|
|
||||||
|
|
||||||
private lateinit var mockMvc: MockMvc
|
|
||||||
|
|
||||||
private lateinit var requestProcessor: RequestProcessor
|
|
||||||
|
|
||||||
|
|
||||||
@BeforeEach
|
|
||||||
fun setup(
|
|
||||||
@Mock requestProcessor: RequestProcessor
|
|
||||||
) {
|
|
||||||
this.requestProcessor = requestProcessor
|
|
||||||
val controller = MtbFileRestController(
|
|
||||||
requestProcessor,
|
|
||||||
ConsentByMtbFile()
|
|
||||||
)
|
|
||||||
this.mockMvc = MockMvcBuilders.standaloneSetup(controller).build()
|
|
||||||
}
|
|
||||||
|
|
||||||
@Test
|
|
||||||
fun shouldProcessPostRequest() {
|
|
||||||
mockMvc.post("/mtbfile") {
|
|
||||||
content = objectMapper.writeValueAsString(bwhcMtbFileContent(Status.ACTIVE))
|
|
||||||
contentType = MediaType.APPLICATION_JSON
|
|
||||||
}.andExpect {
|
|
||||||
status {
|
|
||||||
isAccepted()
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
verify(requestProcessor, times(1)).processMtbFile(any<MtbFile>())
|
|
||||||
}
|
|
||||||
|
|
||||||
@Test
|
|
||||||
fun shouldProcessPostRequestWithRejectedConsent() {
|
|
||||||
mockMvc.post("/mtbfile") {
|
|
||||||
content = objectMapper.writeValueAsString(bwhcMtbFileContent(Status.REJECTED))
|
|
||||||
contentType = MediaType.APPLICATION_JSON
|
|
||||||
}.andExpect {
|
|
||||||
status {
|
|
||||||
isAccepted()
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
verify(requestProcessor, times(1)).processDeletion(
|
|
||||||
anyValueClass(),
|
|
||||||
org.mockito.kotlin.eq(TtpConsentStatus.BROAD_CONSENT_MISSING_OR_REJECTED)
|
|
||||||
)
|
|
||||||
}
|
|
||||||
|
|
||||||
@Test
|
|
||||||
fun shouldProcessDeleteRequest() {
|
|
||||||
mockMvc.delete("/mtbfile/TEST_12345678").andExpect {
|
|
||||||
status {
|
|
||||||
isAccepted()
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
verify(requestProcessor, times(1)).processDeletion(
|
|
||||||
anyValueClass(),
|
|
||||||
org.mockito.kotlin.eq(TtpConsentStatus.UNKNOWN_CHECK_FILE)
|
|
||||||
)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
@Nested
|
|
||||||
inner class BwhcRequestsCheckConsentViaTtp {
|
|
||||||
|
|
||||||
private lateinit var mockMvc: MockMvc
|
|
||||||
|
|
||||||
private lateinit var requestProcessor: RequestProcessor
|
|
||||||
|
|
||||||
private lateinit var gicsConsentService: GicsConsentService
|
|
||||||
|
|
||||||
@BeforeEach
|
|
||||||
fun setup(
|
|
||||||
@Mock requestProcessor: RequestProcessor,
|
|
||||||
@Mock gicsConsentService: GicsConsentService
|
|
||||||
) {
|
|
||||||
this.requestProcessor = requestProcessor
|
|
||||||
val controller = MtbFileRestController(requestProcessor, gicsConsentService)
|
|
||||||
this.mockMvc = MockMvcBuilders.standaloneSetup(controller).build()
|
|
||||||
this.gicsConsentService = gicsConsentService
|
|
||||||
}
|
|
||||||
|
|
||||||
@ParameterizedTest
|
|
||||||
@ValueSource(strings = ["ACTIVE", "REJECTED"])
|
|
||||||
fun shouldProcessPostRequest(status: String) {
|
|
||||||
|
|
||||||
whenever(gicsConsentService.getTtpBroadConsentStatus(any())).thenReturn(TtpConsentStatus.BROAD_CONSENT_GIVEN)
|
|
||||||
|
|
||||||
mockMvc.post("/mtbfile") {
|
|
||||||
content = objectMapper.writeValueAsString(bwhcMtbFileContent(Status.valueOf(status)))
|
|
||||||
contentType = MediaType.APPLICATION_JSON
|
|
||||||
}.andExpect {
|
|
||||||
status {
|
|
||||||
isAccepted()
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
verify(requestProcessor, times(1)).processMtbFile(any<MtbFile>())
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
@ParameterizedTest
|
|
||||||
@ValueSource(strings = ["ACTIVE", "REJECTED"])
|
|
||||||
fun shouldProcessPostRequestWithRejectedConsent(status: String) {
|
|
||||||
|
|
||||||
whenever(gicsConsentService.getTtpBroadConsentStatus(any())).thenReturn(TtpConsentStatus.BROAD_CONSENT_MISSING_OR_REJECTED)
|
|
||||||
|
|
||||||
mockMvc.post("/mtbfile") {
|
|
||||||
content = objectMapper.writeValueAsString(bwhcMtbFileContent(Status.valueOf(status)))
|
|
||||||
contentType = MediaType.APPLICATION_JSON
|
|
||||||
}.andExpect {
|
|
||||||
status {
|
|
||||||
isAccepted()
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// consent status from ttp should override file consent value
|
|
||||||
verify(requestProcessor, times(1)).processDeletion(
|
|
||||||
anyValueClass(),
|
|
||||||
org.mockito.kotlin.eq(TtpConsentStatus.BROAD_CONSENT_MISSING_OR_REJECTED)
|
|
||||||
)
|
|
||||||
}
|
|
||||||
|
|
||||||
@Test
|
|
||||||
fun shouldProcessDeleteRequest() {
|
|
||||||
|
|
||||||
mockMvc.delete("/mtbfile/TEST_12345678").andExpect {
|
|
||||||
status {
|
|
||||||
isAccepted()
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
verify(requestProcessor, times(1)).processDeletion(
|
|
||||||
anyValueClass(),
|
|
||||||
org.mockito.kotlin.eq(TtpConsentStatus.UNKNOWN_CHECK_FILE)
|
|
||||||
)
|
|
||||||
verify(gicsConsentService, times(0)).getTtpBroadConsentStatus(any())
|
|
||||||
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
@Nested
|
|
||||||
inner class BwhcRequestsWithAlias {
|
|
||||||
|
|
||||||
private lateinit var mockMvc: MockMvc
|
|
||||||
|
|
||||||
private lateinit var requestProcessor: RequestProcessor
|
|
||||||
|
|
||||||
@BeforeEach
|
|
||||||
fun setup(
|
|
||||||
@Mock requestProcessor: RequestProcessor
|
|
||||||
) {
|
|
||||||
this.requestProcessor = requestProcessor
|
|
||||||
val controller = MtbFileRestController(
|
|
||||||
requestProcessor,
|
|
||||||
ConsentByMtbFile()
|
|
||||||
)
|
|
||||||
this.mockMvc = MockMvcBuilders.standaloneSetup(controller).build()
|
|
||||||
}
|
|
||||||
|
|
||||||
@Test
|
|
||||||
fun shouldProcessPostRequest() {
|
|
||||||
mockMvc.post("/mtb") {
|
|
||||||
content = objectMapper.writeValueAsString(bwhcMtbFileContent(Status.ACTIVE))
|
|
||||||
contentType = MediaType.APPLICATION_JSON
|
|
||||||
}.andExpect {
|
|
||||||
status {
|
|
||||||
isAccepted()
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
verify(requestProcessor, times(1)).processMtbFile(any<MtbFile>())
|
|
||||||
}
|
|
||||||
|
|
||||||
@Test
|
|
||||||
fun shouldProcessPostRequestWithRejectedConsent() {
|
|
||||||
mockMvc.post("/mtb") {
|
|
||||||
content = objectMapper.writeValueAsString(bwhcMtbFileContent(Status.REJECTED))
|
|
||||||
contentType = MediaType.APPLICATION_JSON
|
|
||||||
}.andExpect {
|
|
||||||
status {
|
|
||||||
isAccepted()
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
verify(requestProcessor, times(1)).processDeletion(
|
|
||||||
anyValueClass(), org.mockito.kotlin.eq(
|
|
||||||
TtpConsentStatus.BROAD_CONSENT_MISSING_OR_REJECTED
|
|
||||||
)
|
|
||||||
)
|
|
||||||
}
|
|
||||||
|
|
||||||
@Test
|
|
||||||
fun shouldProcessDeleteRequest() {
|
|
||||||
mockMvc.delete("/mtb/TEST_12345678").andExpect {
|
|
||||||
status {
|
|
||||||
isAccepted()
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
verify(requestProcessor, times(1)).processDeletion(
|
|
||||||
anyValueClass(), org.mockito.kotlin.eq(
|
|
||||||
TtpConsentStatus.UNKNOWN_CHECK_FILE
|
|
||||||
)
|
|
||||||
)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
@Nested
|
@Nested
|
||||||
inner class RequestsForDnpmDataModel21 {
|
inner class RequestsForDnpmDataModel21 {
|
||||||
|
|
||||||
@@ -304,15 +81,4 @@ class MtbFileRestControllerTest {
|
|||||||
}
|
}
|
||||||
|
|
||||||
}
|
}
|
||||||
|
|
||||||
companion object {
|
|
||||||
fun bwhcMtbFileContent(consentStatus: Status) = MtbFile.builder().withPatient(
|
|
||||||
Patient.builder().withId("TEST_12345678").withBirthDate("2000-08-08").withGender(Patient.Gender.MALE)
|
|
||||||
.build()
|
|
||||||
).withConsent(
|
|
||||||
Consent.builder().withId("1").withStatus(consentStatus).withPatient("TEST_12345678").build()
|
|
||||||
).withEpisode(
|
|
||||||
Episode.builder().withId("1").withPatient("TEST_12345678").withPeriod(PeriodStart("2023-08-08")).build()
|
|
||||||
).build()
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
|
@@ -20,8 +20,6 @@
|
|||||||
package dev.dnpm.etl.processor.output
|
package dev.dnpm.etl.processor.output
|
||||||
|
|
||||||
import com.fasterxml.jackson.databind.ObjectMapper
|
import com.fasterxml.jackson.databind.ObjectMapper
|
||||||
import de.ukw.ccc.bwhc.dto.*
|
|
||||||
import de.ukw.ccc.bwhc.dto.Patient
|
|
||||||
import dev.dnpm.etl.processor.CustomMediaType
|
import dev.dnpm.etl.processor.CustomMediaType
|
||||||
import dev.dnpm.etl.processor.PatientPseudonym
|
import dev.dnpm.etl.processor.PatientPseudonym
|
||||||
import dev.dnpm.etl.processor.RequestId
|
import dev.dnpm.etl.processor.RequestId
|
||||||
@@ -39,7 +37,6 @@ import org.junit.jupiter.params.provider.MethodSource
|
|||||||
import org.mockito.Mock
|
import org.mockito.Mock
|
||||||
import org.mockito.junit.jupiter.MockitoExtension
|
import org.mockito.junit.jupiter.MockitoExtension
|
||||||
import org.mockito.kotlin.*
|
import org.mockito.kotlin.*
|
||||||
import org.springframework.http.MediaType
|
|
||||||
import org.springframework.kafka.core.KafkaTemplate
|
import org.springframework.kafka.core.KafkaTemplate
|
||||||
import org.springframework.kafka.support.SendResult
|
import org.springframework.kafka.support.SendResult
|
||||||
import org.springframework.retry.policy.SimpleRetryPolicy
|
import org.springframework.retry.policy.SimpleRetryPolicy
|
||||||
@@ -74,20 +71,6 @@ class KafkaMtbFileSenderTest {
|
|||||||
this.kafkaMtbFileSender = KafkaMtbFileSender(kafkaTemplate, kafkaProperties, retryTemplate, objectMapper)
|
this.kafkaMtbFileSender = KafkaMtbFileSender(kafkaTemplate, kafkaProperties, retryTemplate, objectMapper)
|
||||||
}
|
}
|
||||||
|
|
||||||
@ParameterizedTest
|
|
||||||
@MethodSource("dev.dnpm.etl.processor.output.KafkaMtbFileSenderTest#requestWithResponseSource")
|
|
||||||
fun shouldSendMtbFileRequestAndReturnExpectedState(testData: TestData) {
|
|
||||||
doAnswer {
|
|
||||||
if (null != testData.exception) {
|
|
||||||
throw testData.exception
|
|
||||||
}
|
|
||||||
completedFuture(SendResult<String, String>(null, null))
|
|
||||||
}.whenever(kafkaTemplate).send(any<ProducerRecord<String, String>>())
|
|
||||||
|
|
||||||
val response = kafkaMtbFileSender.send(BwhcV1MtbFileRequest(TEST_REQUEST_ID, bwhcV1MtbFile(Consent.Status.ACTIVE)))
|
|
||||||
assertThat(response.status).isEqualTo(testData.requestStatus)
|
|
||||||
}
|
|
||||||
|
|
||||||
@ParameterizedTest
|
@ParameterizedTest
|
||||||
@MethodSource("dev.dnpm.etl.processor.output.KafkaMtbFileSenderTest#requestWithResponseSource")
|
@MethodSource("dev.dnpm.etl.processor.output.KafkaMtbFileSenderTest#requestWithResponseSource")
|
||||||
fun shouldSendDeleteRequestAndReturnExpectedState(testData: TestData) {
|
fun shouldSendDeleteRequestAndReturnExpectedState(testData: TestData) {
|
||||||
@@ -102,66 +85,6 @@ class KafkaMtbFileSenderTest {
|
|||||||
assertThat(response.status).isEqualTo(testData.requestStatus)
|
assertThat(response.status).isEqualTo(testData.requestStatus)
|
||||||
}
|
}
|
||||||
|
|
||||||
@Test
|
|
||||||
fun shouldSendMtbFileRequestWithCorrectKeyAndHeaderAndBody() {
|
|
||||||
doAnswer {
|
|
||||||
completedFuture(SendResult<String, String>(null, null))
|
|
||||||
}.whenever(kafkaTemplate).send(any<ProducerRecord<String, String>>())
|
|
||||||
|
|
||||||
kafkaMtbFileSender.send(BwhcV1MtbFileRequest(TEST_REQUEST_ID, bwhcV1MtbFile(Consent.Status.ACTIVE)))
|
|
||||||
|
|
||||||
val captor = argumentCaptor<ProducerRecord<String, String>>()
|
|
||||||
verify(kafkaTemplate, times(1)).send(captor.capture())
|
|
||||||
assertThat(captor.firstValue.key()).isNotNull
|
|
||||||
assertThat(captor.firstValue.key()).isEqualTo("{\"pid\": \"PID\"}")
|
|
||||||
assertThat(captor.firstValue.headers().headers("contentType")).isNotNull
|
|
||||||
assertThat(captor.firstValue.headers().headers("contentType")?.firstOrNull()?.value()).isEqualTo(MediaType.APPLICATION_JSON_VALUE.toByteArray())
|
|
||||||
assertThat(captor.firstValue.value()).isNotNull
|
|
||||||
assertThat(captor.firstValue.value()).isEqualTo(objectMapper.writeValueAsString(bwhcV1kafkaRecordData(TEST_REQUEST_ID, Consent.Status.ACTIVE)))
|
|
||||||
}
|
|
||||||
|
|
||||||
@Test
|
|
||||||
fun shouldSendDeleteRequestWithCorrectKeyAndBody() {
|
|
||||||
doAnswer {
|
|
||||||
completedFuture(SendResult<String, String>(null, null))
|
|
||||||
}.whenever(kafkaTemplate).send(any<ProducerRecord<String, String>>())
|
|
||||||
|
|
||||||
kafkaMtbFileSender.send(DeleteRequest(TEST_REQUEST_ID, TEST_PATIENT_PSEUDONYM))
|
|
||||||
|
|
||||||
val captor = argumentCaptor<ProducerRecord<String, String>>()
|
|
||||||
verify(kafkaTemplate, times(1)).send(captor.capture())
|
|
||||||
assertThat(captor.firstValue.key()).isNotNull
|
|
||||||
assertThat(captor.firstValue.key()).isEqualTo("{\"pid\": \"PID\"}")
|
|
||||||
assertThat(captor.firstValue.value()).isNotNull
|
|
||||||
assertThat(captor.firstValue.value()).isEqualTo(objectMapper.writeValueAsString(bwhcV1kafkaRecordData(TEST_REQUEST_ID, Consent.Status.REJECTED)))
|
|
||||||
}
|
|
||||||
|
|
||||||
@ParameterizedTest
|
|
||||||
@MethodSource("dev.dnpm.etl.processor.output.KafkaMtbFileSenderTest#requestWithResponseSource")
|
|
||||||
fun shouldRetryOnMtbFileKafkaSendError(testData: TestData) {
|
|
||||||
val kafkaProperties = KafkaProperties("testtopic")
|
|
||||||
val retryTemplate = RetryTemplateBuilder().customPolicy(SimpleRetryPolicy(3)).build()
|
|
||||||
this.kafkaMtbFileSender = KafkaMtbFileSender(this.kafkaTemplate, kafkaProperties, retryTemplate, this.objectMapper)
|
|
||||||
|
|
||||||
doAnswer {
|
|
||||||
if (null != testData.exception) {
|
|
||||||
throw testData.exception
|
|
||||||
}
|
|
||||||
completedFuture(SendResult<String, String>(null, null))
|
|
||||||
}.whenever(kafkaTemplate).send(any<ProducerRecord<String, String>>())
|
|
||||||
|
|
||||||
kafkaMtbFileSender.send(BwhcV1MtbFileRequest(TEST_REQUEST_ID, bwhcV1MtbFile(Consent.Status.ACTIVE)))
|
|
||||||
|
|
||||||
val expectedCount = when (testData.exception) {
|
|
||||||
// OK - No Retry
|
|
||||||
null -> times(1)
|
|
||||||
// Request failed - Retry max 3 times
|
|
||||||
else -> times(3)
|
|
||||||
}
|
|
||||||
|
|
||||||
verify(kafkaTemplate, expectedCount).send(any<ProducerRecord<String, String>>())
|
|
||||||
}
|
|
||||||
|
|
||||||
@ParameterizedTest
|
@ParameterizedTest
|
||||||
@MethodSource("dev.dnpm.etl.processor.output.KafkaMtbFileSenderTest#requestWithResponseSource")
|
@MethodSource("dev.dnpm.etl.processor.output.KafkaMtbFileSenderTest#requestWithResponseSource")
|
||||||
fun shouldRetryOnDeleteKafkaSendError(testData: TestData) {
|
fun shouldRetryOnDeleteKafkaSendError(testData: TestData) {
|
||||||
@@ -276,41 +199,6 @@ class KafkaMtbFileSenderTest {
|
|||||||
val TEST_REQUEST_ID = RequestId("TestId")
|
val TEST_REQUEST_ID = RequestId("TestId")
|
||||||
val TEST_PATIENT_PSEUDONYM = PatientPseudonym("PID")
|
val TEST_PATIENT_PSEUDONYM = PatientPseudonym("PID")
|
||||||
|
|
||||||
fun bwhcV1MtbFile(consentStatus: Consent.Status): MtbFile {
|
|
||||||
return if (consentStatus == Consent.Status.ACTIVE) {
|
|
||||||
MtbFile.builder()
|
|
||||||
.withPatient(
|
|
||||||
Patient.builder()
|
|
||||||
.withId("PID")
|
|
||||||
.withBirthDate("2000-08-08")
|
|
||||||
.withGender(Patient.Gender.MALE)
|
|
||||||
.build()
|
|
||||||
)
|
|
||||||
.withConsent(
|
|
||||||
Consent.builder()
|
|
||||||
.withId("1")
|
|
||||||
.withStatus(consentStatus)
|
|
||||||
.withPatient("PID")
|
|
||||||
.build()
|
|
||||||
)
|
|
||||||
.withEpisode(
|
|
||||||
Episode.builder()
|
|
||||||
.withId("1")
|
|
||||||
.withPatient("PID")
|
|
||||||
.withPeriod(PeriodStart("2023-08-08"))
|
|
||||||
.build()
|
|
||||||
)
|
|
||||||
} else {
|
|
||||||
MtbFile.builder()
|
|
||||||
.withConsent(
|
|
||||||
Consent.builder()
|
|
||||||
.withStatus(consentStatus)
|
|
||||||
.withPatient("PID")
|
|
||||||
.build()
|
|
||||||
)
|
|
||||||
}.build()
|
|
||||||
}
|
|
||||||
|
|
||||||
fun dnpmV2MtbFile(): Mtb {
|
fun dnpmV2MtbFile(): Mtb {
|
||||||
return Mtb().apply {
|
return Mtb().apply {
|
||||||
this.patient = dev.pcvolkmer.mv64e.mtb.Patient().apply {
|
this.patient = dev.pcvolkmer.mv64e.mtb.Patient().apply {
|
||||||
@@ -334,10 +222,6 @@ class KafkaMtbFileSenderTest {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
fun bwhcV1kafkaRecordData(requestId: RequestId, consentStatus: Consent.Status): MtbRequest {
|
|
||||||
return BwhcV1MtbFileRequest(requestId, bwhcV1MtbFile(consentStatus))
|
|
||||||
}
|
|
||||||
|
|
||||||
fun dnmpV2kafkaRecordData(requestId: RequestId): MtbRequest {
|
fun dnmpV2kafkaRecordData(requestId: RequestId): MtbRequest {
|
||||||
return DnpmV2MtbFileRequest(requestId, dnpmV2MtbFile())
|
return DnpmV2MtbFileRequest(requestId, dnpmV2MtbFile())
|
||||||
}
|
}
|
||||||
|
@@ -1,313 +0,0 @@
|
|||||||
/*
|
|
||||||
* This file is part of ETL-Processor
|
|
||||||
*
|
|
||||||
* Copyright (c) 2025 Comprehensive Cancer Center Mainfranken, Datenintegrationszentrum Philipps-Universität Marburg and Contributors
|
|
||||||
*
|
|
||||||
* This program is free software: you can redistribute it and/or modify
|
|
||||||
* it under the terms of the GNU Affero General Public License as published
|
|
||||||
* by the Free Software Foundation, either version 3 of the License, or
|
|
||||||
* (at your option) any later version.
|
|
||||||
*
|
|
||||||
* This program is distributed in the hope that it will be useful,
|
|
||||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of
|
|
||||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
|
||||||
* GNU Affero General Public License for more details.
|
|
||||||
*
|
|
||||||
* You should have received a copy of the GNU Affero General Public License
|
|
||||||
* along with this program. If not, see <https://www.gnu.org/licenses/>.
|
|
||||||
*/
|
|
||||||
|
|
||||||
package dev.dnpm.etl.processor.output
|
|
||||||
|
|
||||||
import com.fasterxml.jackson.databind.ObjectMapper
|
|
||||||
import com.fasterxml.jackson.module.kotlin.KotlinModule
|
|
||||||
import de.ukw.ccc.bwhc.dto.*
|
|
||||||
import dev.dnpm.etl.processor.PatientPseudonym
|
|
||||||
import dev.dnpm.etl.processor.RequestId
|
|
||||||
import dev.dnpm.etl.processor.config.RestTargetProperties
|
|
||||||
import dev.dnpm.etl.processor.monitoring.ReportService
|
|
||||||
import dev.dnpm.etl.processor.monitoring.RequestStatus
|
|
||||||
import org.assertj.core.api.Assertions.assertThat
|
|
||||||
import org.junit.jupiter.api.BeforeEach
|
|
||||||
import org.junit.jupiter.params.ParameterizedTest
|
|
||||||
import org.junit.jupiter.params.provider.MethodSource
|
|
||||||
import org.springframework.http.HttpHeaders
|
|
||||||
import org.springframework.http.HttpMethod
|
|
||||||
import org.springframework.http.HttpStatus
|
|
||||||
import org.springframework.http.MediaType
|
|
||||||
import org.springframework.retry.policy.SimpleRetryPolicy
|
|
||||||
import org.springframework.retry.support.RetryTemplateBuilder
|
|
||||||
import org.springframework.test.web.client.ExpectedCount
|
|
||||||
import org.springframework.test.web.client.MockRestServiceServer
|
|
||||||
import org.springframework.test.web.client.match.MockRestRequestMatchers.*
|
|
||||||
import org.springframework.test.web.client.response.MockRestResponseCreators.withStatus
|
|
||||||
import org.springframework.web.client.RestTemplate
|
|
||||||
|
|
||||||
class RestBwhcMtbFileSenderTest {
|
|
||||||
|
|
||||||
private lateinit var mockRestServiceServer: MockRestServiceServer
|
|
||||||
|
|
||||||
private lateinit var restMtbFileSender: RestMtbFileSender
|
|
||||||
|
|
||||||
private var reportService = ReportService(ObjectMapper().registerModule(KotlinModule.Builder().build()))
|
|
||||||
|
|
||||||
@BeforeEach
|
|
||||||
fun setup() {
|
|
||||||
val restTemplate = RestTemplate()
|
|
||||||
val restTargetProperties = RestTargetProperties("http://localhost:9000/mtbfile", null, null)
|
|
||||||
val retryTemplate = RetryTemplateBuilder().customPolicy(SimpleRetryPolicy(1)).build()
|
|
||||||
|
|
||||||
this.mockRestServiceServer = MockRestServiceServer.createServer(restTemplate)
|
|
||||||
|
|
||||||
this.restMtbFileSender =
|
|
||||||
RestBwhcMtbFileSender(restTemplate, restTargetProperties, retryTemplate, reportService)
|
|
||||||
}
|
|
||||||
|
|
||||||
@ParameterizedTest
|
|
||||||
@MethodSource("deleteRequestWithResponseSource")
|
|
||||||
fun shouldReturnExpectedResponseForDelete(requestWithResponse: RequestWithResponse) {
|
|
||||||
this.mockRestServiceServer
|
|
||||||
.expect(method(HttpMethod.DELETE))
|
|
||||||
.andExpect(requestTo("http://localhost:9000/mtbfile/Patient/${TEST_PATIENT_PSEUDONYM.value}"))
|
|
||||||
.andRespond {
|
|
||||||
withStatus(requestWithResponse.httpStatus).body(requestWithResponse.body).createResponse(it)
|
|
||||||
}
|
|
||||||
|
|
||||||
val response = restMtbFileSender.send(DeleteRequest(TEST_REQUEST_ID, TEST_PATIENT_PSEUDONYM))
|
|
||||||
assertThat(response.status).isEqualTo(requestWithResponse.response.status)
|
|
||||||
assertThat(response.body).isEqualTo(requestWithResponse.response.body)
|
|
||||||
}
|
|
||||||
|
|
||||||
@ParameterizedTest
|
|
||||||
@MethodSource("mtbFileRequestWithResponseSource")
|
|
||||||
fun shouldReturnExpectedResponseForMtbFilePost(requestWithResponse: RequestWithResponse) {
|
|
||||||
this.mockRestServiceServer
|
|
||||||
.expect(method(HttpMethod.POST))
|
|
||||||
.andExpect(requestTo("http://localhost:9000/mtbfile/MTBFile"))
|
|
||||||
.andExpect(header(HttpHeaders.CONTENT_TYPE, MediaType.APPLICATION_JSON_VALUE))
|
|
||||||
.andRespond {
|
|
||||||
withStatus(requestWithResponse.httpStatus).body(requestWithResponse.body).createResponse(it)
|
|
||||||
}
|
|
||||||
|
|
||||||
val response = restMtbFileSender.send(BwhcV1MtbFileRequest(TEST_REQUEST_ID, mtbFile))
|
|
||||||
assertThat(response.status).isEqualTo(requestWithResponse.response.status)
|
|
||||||
assertThat(response.body).isEqualTo(requestWithResponse.response.body)
|
|
||||||
}
|
|
||||||
|
|
||||||
@ParameterizedTest
|
|
||||||
@MethodSource("mtbFileRequestWithResponseSource")
|
|
||||||
fun shouldRetryOnMtbFileHttpRequestError(requestWithResponse: RequestWithResponse) {
|
|
||||||
val restTemplate = RestTemplate()
|
|
||||||
val restTargetProperties = RestTargetProperties("http://localhost:9000/mtbfile", null, null)
|
|
||||||
val retryTemplate = RetryTemplateBuilder().customPolicy(SimpleRetryPolicy(3)).build()
|
|
||||||
|
|
||||||
this.mockRestServiceServer = MockRestServiceServer.createServer(restTemplate)
|
|
||||||
this.restMtbFileSender =
|
|
||||||
RestBwhcMtbFileSender(restTemplate, restTargetProperties, retryTemplate, reportService)
|
|
||||||
|
|
||||||
val expectedCount = when (requestWithResponse.httpStatus) {
|
|
||||||
// OK - No Retry
|
|
||||||
HttpStatus.OK, HttpStatus.CREATED -> ExpectedCount.max(1)
|
|
||||||
// Request failed - Retry max 3 times
|
|
||||||
else -> ExpectedCount.max(3)
|
|
||||||
}
|
|
||||||
|
|
||||||
this.mockRestServiceServer
|
|
||||||
.expect(expectedCount, method(HttpMethod.POST))
|
|
||||||
.andExpect(requestTo("http://localhost:9000/mtbfile/MTBFile"))
|
|
||||||
.andRespond {
|
|
||||||
withStatus(requestWithResponse.httpStatus).body(requestWithResponse.body).createResponse(it)
|
|
||||||
}
|
|
||||||
|
|
||||||
val response = restMtbFileSender.send(BwhcV1MtbFileRequest(TEST_REQUEST_ID, mtbFile))
|
|
||||||
assertThat(response.status).isEqualTo(requestWithResponse.response.status)
|
|
||||||
assertThat(response.body).isEqualTo(requestWithResponse.response.body)
|
|
||||||
}
|
|
||||||
|
|
||||||
@ParameterizedTest
|
|
||||||
@MethodSource("deleteRequestWithResponseSource")
|
|
||||||
fun shouldRetryOnDeleteHttpRequestError(requestWithResponse: RequestWithResponse) {
|
|
||||||
val restTemplate = RestTemplate()
|
|
||||||
val restTargetProperties = RestTargetProperties("http://localhost:9000/mtbfile", null, null)
|
|
||||||
val retryTemplate = RetryTemplateBuilder().customPolicy(SimpleRetryPolicy(3)).build()
|
|
||||||
|
|
||||||
this.mockRestServiceServer = MockRestServiceServer.createServer(restTemplate)
|
|
||||||
this.restMtbFileSender =
|
|
||||||
RestBwhcMtbFileSender(restTemplate, restTargetProperties, retryTemplate, reportService)
|
|
||||||
|
|
||||||
val expectedCount = when (requestWithResponse.httpStatus) {
|
|
||||||
// OK - No Retry
|
|
||||||
HttpStatus.OK, HttpStatus.CREATED -> ExpectedCount.max(1)
|
|
||||||
// Request failed - Retry max 3 times
|
|
||||||
else -> ExpectedCount.max(3)
|
|
||||||
}
|
|
||||||
|
|
||||||
this.mockRestServiceServer
|
|
||||||
.expect(expectedCount, method(HttpMethod.DELETE))
|
|
||||||
.andExpect(requestTo("http://localhost:9000/mtbfile/Patient/${TEST_PATIENT_PSEUDONYM.value}"))
|
|
||||||
.andRespond {
|
|
||||||
withStatus(requestWithResponse.httpStatus).body(requestWithResponse.body).createResponse(it)
|
|
||||||
}
|
|
||||||
|
|
||||||
val response = restMtbFileSender.send(DeleteRequest(TEST_REQUEST_ID, TEST_PATIENT_PSEUDONYM))
|
|
||||||
assertThat(response.status).isEqualTo(requestWithResponse.response.status)
|
|
||||||
assertThat(response.body).isEqualTo(requestWithResponse.response.body)
|
|
||||||
}
|
|
||||||
|
|
||||||
companion object {
|
|
||||||
data class RequestWithResponse(
|
|
||||||
val httpStatus: HttpStatus,
|
|
||||||
val body: String,
|
|
||||||
val response: MtbFileSender.Response
|
|
||||||
)
|
|
||||||
|
|
||||||
val TEST_REQUEST_ID = RequestId("TestId")
|
|
||||||
val TEST_PATIENT_PSEUDONYM = PatientPseudonym("PID")
|
|
||||||
|
|
||||||
val mtbFile: MtbFile = MtbFile.builder()
|
|
||||||
.withPatient(
|
|
||||||
Patient.builder()
|
|
||||||
.withId("PID")
|
|
||||||
.withBirthDate("2000-08-08")
|
|
||||||
.withGender(Patient.Gender.MALE)
|
|
||||||
.build()
|
|
||||||
)
|
|
||||||
.withConsent(
|
|
||||||
Consent.builder()
|
|
||||||
.withId("1")
|
|
||||||
.withStatus(Consent.Status.ACTIVE)
|
|
||||||
.withPatient("PID")
|
|
||||||
.build()
|
|
||||||
)
|
|
||||||
.withEpisode(
|
|
||||||
Episode.builder()
|
|
||||||
.withId("1")
|
|
||||||
.withPatient("PID")
|
|
||||||
.withPeriod(PeriodStart("2023-08-08"))
|
|
||||||
.build()
|
|
||||||
)
|
|
||||||
.build()
|
|
||||||
|
|
||||||
private const val ERROR_RESPONSE_BODY = "Sonstiger Fehler bei der Übertragung"
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Synthetic http responses with related request status
|
|
||||||
* Also see: https://ibmi-intra.cs.uni-tuebingen.de/display/ZPM/bwHC+REST+API
|
|
||||||
*/
|
|
||||||
@JvmStatic
|
|
||||||
fun mtbFileRequestWithResponseSource(): Set<RequestWithResponse> {
|
|
||||||
return setOf(
|
|
||||||
RequestWithResponse(
|
|
||||||
HttpStatus.OK,
|
|
||||||
responseBodyWithMaxSeverity(ReportService.Severity.INFO),
|
|
||||||
MtbFileSender.Response(
|
|
||||||
RequestStatus.SUCCESS,
|
|
||||||
responseBodyWithMaxSeverity(ReportService.Severity.INFO)
|
|
||||||
)
|
|
||||||
),
|
|
||||||
RequestWithResponse(
|
|
||||||
HttpStatus.CREATED,
|
|
||||||
responseBodyWithMaxSeverity(ReportService.Severity.WARNING),
|
|
||||||
MtbFileSender.Response(
|
|
||||||
RequestStatus.WARNING,
|
|
||||||
responseBodyWithMaxSeverity(ReportService.Severity.WARNING)
|
|
||||||
)
|
|
||||||
),
|
|
||||||
RequestWithResponse(
|
|
||||||
HttpStatus.BAD_REQUEST,
|
|
||||||
responseBodyWithMaxSeverity(ReportService.Severity.ERROR),
|
|
||||||
MtbFileSender.Response(RequestStatus.ERROR, responseBodyWithMaxSeverity(ReportService.Severity.ERROR))
|
|
||||||
),
|
|
||||||
RequestWithResponse(
|
|
||||||
HttpStatus.UNPROCESSABLE_ENTITY,
|
|
||||||
responseBodyWithMaxSeverity(ReportService.Severity.FATAL),
|
|
||||||
MtbFileSender.Response(
|
|
||||||
RequestStatus.ERROR,
|
|
||||||
responseBodyWithMaxSeverity(ReportService.Severity.FATAL)
|
|
||||||
)
|
|
||||||
),
|
|
||||||
// Some more errors not mentioned in documentation
|
|
||||||
RequestWithResponse(
|
|
||||||
HttpStatus.NOT_FOUND,
|
|
||||||
ERROR_RESPONSE_BODY,
|
|
||||||
MtbFileSender.Response(RequestStatus.ERROR, ERROR_RESPONSE_BODY)
|
|
||||||
),
|
|
||||||
RequestWithResponse(
|
|
||||||
HttpStatus.INTERNAL_SERVER_ERROR,
|
|
||||||
ERROR_RESPONSE_BODY,
|
|
||||||
MtbFileSender.Response(RequestStatus.ERROR, ERROR_RESPONSE_BODY)
|
|
||||||
)
|
|
||||||
)
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Synthetic http responses with related request status
|
|
||||||
* Also see: https://ibmi-intra.cs.uni-tuebingen.de/display/ZPM/bwHC+REST+API
|
|
||||||
*/
|
|
||||||
@JvmStatic
|
|
||||||
fun deleteRequestWithResponseSource(): Set<RequestWithResponse> {
|
|
||||||
return setOf(
|
|
||||||
RequestWithResponse(HttpStatus.OK, "", MtbFileSender.Response(RequestStatus.SUCCESS)),
|
|
||||||
// Some more errors not mentioned in documentation
|
|
||||||
RequestWithResponse(
|
|
||||||
HttpStatus.NOT_FOUND,
|
|
||||||
"what????",
|
|
||||||
MtbFileSender.Response(RequestStatus.ERROR, ERROR_RESPONSE_BODY)
|
|
||||||
),
|
|
||||||
RequestWithResponse(
|
|
||||||
HttpStatus.INTERNAL_SERVER_ERROR,
|
|
||||||
"what????",
|
|
||||||
MtbFileSender.Response(RequestStatus.ERROR, ERROR_RESPONSE_BODY)
|
|
||||||
)
|
|
||||||
)
|
|
||||||
}
|
|
||||||
|
|
||||||
fun responseBodyWithMaxSeverity(severity: ReportService.Severity): String {
|
|
||||||
return when (severity) {
|
|
||||||
ReportService.Severity.INFO -> """
|
|
||||||
{
|
|
||||||
"patient": "PID",
|
|
||||||
"issues": [
|
|
||||||
{ "severity": "info", "message": "Info Message" }
|
|
||||||
]
|
|
||||||
}
|
|
||||||
"""
|
|
||||||
|
|
||||||
ReportService.Severity.WARNING -> """
|
|
||||||
{
|
|
||||||
"patient": "PID",
|
|
||||||
"issues": [
|
|
||||||
{ "severity": "info", "message": "Info Message" },
|
|
||||||
{ "severity": "warning", "message": "Warning Message" }
|
|
||||||
]
|
|
||||||
}
|
|
||||||
"""
|
|
||||||
|
|
||||||
ReportService.Severity.ERROR -> """
|
|
||||||
{
|
|
||||||
"patient": "PID",
|
|
||||||
"issues": [
|
|
||||||
{ "severity": "info", "message": "Info Message" },
|
|
||||||
{ "severity": "warning", "message": "Warning Message" },
|
|
||||||
{ "severity": "error", "message": "Error Message" }
|
|
||||||
]
|
|
||||||
}
|
|
||||||
"""
|
|
||||||
|
|
||||||
ReportService.Severity.FATAL -> """
|
|
||||||
{
|
|
||||||
"patient": "PID",
|
|
||||||
"issues": [
|
|
||||||
{ "severity": "info", "message": "Info Message" },
|
|
||||||
{ "severity": "warning", "message": "Warning Message" },
|
|
||||||
{ "severity": "error", "message": "Error Message" },
|
|
||||||
{ "severity": "fatal", "message": "Fatal Message" }
|
|
||||||
]
|
|
||||||
}
|
|
||||||
"""
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
}
|
|
@@ -21,8 +21,6 @@ package dev.dnpm.etl.processor.output
|
|||||||
|
|
||||||
import com.fasterxml.jackson.databind.ObjectMapper
|
import com.fasterxml.jackson.databind.ObjectMapper
|
||||||
import com.fasterxml.jackson.module.kotlin.KotlinModule
|
import com.fasterxml.jackson.module.kotlin.KotlinModule
|
||||||
import de.ukw.ccc.bwhc.dto.*
|
|
||||||
import de.ukw.ccc.bwhc.dto.Patient
|
|
||||||
import dev.dnpm.etl.processor.CustomMediaType
|
import dev.dnpm.etl.processor.CustomMediaType
|
||||||
import dev.dnpm.etl.processor.PatientPseudonym
|
import dev.dnpm.etl.processor.PatientPseudonym
|
||||||
import dev.dnpm.etl.processor.RequestId
|
import dev.dnpm.etl.processor.RequestId
|
||||||
@@ -54,78 +52,6 @@ import java.util.*
|
|||||||
|
|
||||||
class RestDipMtbFileSenderTest {
|
class RestDipMtbFileSenderTest {
|
||||||
|
|
||||||
@Nested
|
|
||||||
inner class BwhcV1ContentRequest {
|
|
||||||
|
|
||||||
private lateinit var mockRestServiceServer: MockRestServiceServer
|
|
||||||
|
|
||||||
private lateinit var restMtbFileSender: RestMtbFileSender
|
|
||||||
|
|
||||||
private var reportService = ReportService(ObjectMapper().registerModule(KotlinModule.Builder().build()))
|
|
||||||
|
|
||||||
@BeforeEach
|
|
||||||
fun setup() {
|
|
||||||
val restTemplate = RestTemplate()
|
|
||||||
val restTargetProperties = RestTargetProperties("http://localhost:9000/api", null, null, false)
|
|
||||||
val retryTemplate = RetryTemplateBuilder().customPolicy(SimpleRetryPolicy(1)).build()
|
|
||||||
|
|
||||||
this.mockRestServiceServer = MockRestServiceServer.createServer(restTemplate)
|
|
||||||
|
|
||||||
this.restMtbFileSender =
|
|
||||||
RestDipMtbFileSender(restTemplate, restTargetProperties, retryTemplate, reportService)
|
|
||||||
}
|
|
||||||
|
|
||||||
@ParameterizedTest
|
|
||||||
@MethodSource("dev.dnpm.etl.processor.output.RestDipMtbFileSenderTest#mtbFileRequestWithResponseSource")
|
|
||||||
fun shouldReturnExpectedResponseForMtbFilePost(requestWithResponse: RequestWithResponse) {
|
|
||||||
this.mockRestServiceServer
|
|
||||||
.expect(method(HttpMethod.POST))
|
|
||||||
.andExpect(requestTo("http://localhost:9000/api/mtb/etl/patient-record"))
|
|
||||||
.andExpect(header(HttpHeaders.CONTENT_TYPE, MediaType.APPLICATION_JSON_VALUE))
|
|
||||||
.andRespond {
|
|
||||||
withStatus(requestWithResponse.httpStatus).body(requestWithResponse.body).createResponse(it)
|
|
||||||
}
|
|
||||||
|
|
||||||
val response = restMtbFileSender.send(BwhcV1MtbFileRequest(TEST_REQUEST_ID, bwhcV1mtbFile))
|
|
||||||
assertThat(response.status).isEqualTo(requestWithResponse.response.status)
|
|
||||||
assertThat(response.body).isEqualTo(requestWithResponse.response.body)
|
|
||||||
}
|
|
||||||
|
|
||||||
@ParameterizedTest
|
|
||||||
@MethodSource("dev.dnpm.etl.processor.output.RestDipMtbFileSenderTest#mtbFileRequestWithResponseSource")
|
|
||||||
fun shouldRetryOnMtbFileHttpRequestError(requestWithResponse: RequestWithResponse) {
|
|
||||||
val restTemplate = RestTemplate()
|
|
||||||
val restTargetProperties = RestTargetProperties("http://localhost:9000/api", null, null, false)
|
|
||||||
val retryTemplate = AppConfiguration().retryTemplate(AppConfigProperties("http://localhost:9000"))
|
|
||||||
retryTemplate.setBackOffPolicy(NoBackOffPolicy())
|
|
||||||
|
|
||||||
this.mockRestServiceServer = MockRestServiceServer.createServer(restTemplate)
|
|
||||||
this.restMtbFileSender =
|
|
||||||
RestDipMtbFileSender(restTemplate, restTargetProperties, retryTemplate, reportService)
|
|
||||||
|
|
||||||
val expectedCount = when (requestWithResponse.httpStatus) {
|
|
||||||
// OK - No Retry
|
|
||||||
HttpStatus.OK, HttpStatus.CREATED, HttpStatus.UNPROCESSABLE_ENTITY, HttpStatus.BAD_REQUEST -> ExpectedCount.max(
|
|
||||||
1
|
|
||||||
)
|
|
||||||
// Request failed - Retry max 3 times
|
|
||||||
else -> ExpectedCount.max(3)
|
|
||||||
}
|
|
||||||
|
|
||||||
this.mockRestServiceServer
|
|
||||||
.expect(expectedCount, method(HttpMethod.POST))
|
|
||||||
.andExpect(requestTo("http://localhost:9000/api/mtb/etl/patient-record"))
|
|
||||||
.andRespond {
|
|
||||||
withStatus(requestWithResponse.httpStatus).body(requestWithResponse.body).createResponse(it)
|
|
||||||
}
|
|
||||||
|
|
||||||
val response = restMtbFileSender.send(BwhcV1MtbFileRequest(TEST_REQUEST_ID, bwhcV1mtbFile))
|
|
||||||
assertThat(response.status).isEqualTo(requestWithResponse.response.status)
|
|
||||||
assertThat(response.body).isEqualTo(requestWithResponse.response.body)
|
|
||||||
}
|
|
||||||
|
|
||||||
}
|
|
||||||
|
|
||||||
@Nested
|
@Nested
|
||||||
inner class DnpmV2ContentRequest {
|
inner class DnpmV2ContentRequest {
|
||||||
|
|
||||||
@@ -138,7 +64,7 @@ class RestDipMtbFileSenderTest {
|
|||||||
@BeforeEach
|
@BeforeEach
|
||||||
fun setup() {
|
fun setup() {
|
||||||
val restTemplate = RestTemplate()
|
val restTemplate = RestTemplate()
|
||||||
val restTargetProperties = RestTargetProperties("http://localhost:9000/api", null, null, false)
|
val restTargetProperties = RestTargetProperties("http://localhost:9000/api", null, null)
|
||||||
val retryTemplate = RetryTemplateBuilder().customPolicy(SimpleRetryPolicy(1)).build()
|
val retryTemplate = RetryTemplateBuilder().customPolicy(SimpleRetryPolicy(1)).build()
|
||||||
|
|
||||||
this.mockRestServiceServer = MockRestServiceServer.createServer(restTemplate)
|
this.mockRestServiceServer = MockRestServiceServer.createServer(restTemplate)
|
||||||
@@ -176,7 +102,7 @@ class RestDipMtbFileSenderTest {
|
|||||||
@BeforeEach
|
@BeforeEach
|
||||||
fun setup() {
|
fun setup() {
|
||||||
val restTemplate = RestTemplate()
|
val restTemplate = RestTemplate()
|
||||||
val restTargetProperties = RestTargetProperties("http://localhost:9000/api", null, null, false)
|
val restTargetProperties = RestTargetProperties("http://localhost:9000/api", null, null)
|
||||||
val retryTemplate = RetryTemplateBuilder().customPolicy(SimpleRetryPolicy(1)).build()
|
val retryTemplate = RetryTemplateBuilder().customPolicy(SimpleRetryPolicy(1)).build()
|
||||||
|
|
||||||
this.mockRestServiceServer = MockRestServiceServer.createServer(restTemplate)
|
this.mockRestServiceServer = MockRestServiceServer.createServer(restTemplate)
|
||||||
@@ -204,8 +130,8 @@ class RestDipMtbFileSenderTest {
|
|||||||
@MethodSource("dev.dnpm.etl.processor.output.RestDipMtbFileSenderTest#deleteRequestWithResponseSource")
|
@MethodSource("dev.dnpm.etl.processor.output.RestDipMtbFileSenderTest#deleteRequestWithResponseSource")
|
||||||
fun shouldRetryOnDeleteHttpRequestError(requestWithResponse: RequestWithResponse) {
|
fun shouldRetryOnDeleteHttpRequestError(requestWithResponse: RequestWithResponse) {
|
||||||
val restTemplate = RestTemplate()
|
val restTemplate = RestTemplate()
|
||||||
val restTargetProperties = RestTargetProperties("http://localhost:9000/api", null, null, false)
|
val restTargetProperties = RestTargetProperties("http://localhost:9000/api", null, null)
|
||||||
val retryTemplate = AppConfiguration().retryTemplate(AppConfigProperties("http://localhost:9000"))
|
val retryTemplate = AppConfiguration().retryTemplate(AppConfigProperties())
|
||||||
retryTemplate.setBackOffPolicy(NoBackOffPolicy())
|
retryTemplate.setBackOffPolicy(NoBackOffPolicy())
|
||||||
|
|
||||||
this.mockRestServiceServer = MockRestServiceServer.createServer(restTemplate)
|
this.mockRestServiceServer = MockRestServiceServer.createServer(restTemplate)
|
||||||
@@ -245,30 +171,6 @@ class RestDipMtbFileSenderTest {
|
|||||||
val TEST_REQUEST_ID = RequestId("TestId")
|
val TEST_REQUEST_ID = RequestId("TestId")
|
||||||
val TEST_PATIENT_PSEUDONYM = PatientPseudonym("PID")
|
val TEST_PATIENT_PSEUDONYM = PatientPseudonym("PID")
|
||||||
|
|
||||||
val bwhcV1mtbFile: MtbFile = MtbFile.builder()
|
|
||||||
.withPatient(
|
|
||||||
Patient.builder()
|
|
||||||
.withId("PID")
|
|
||||||
.withBirthDate("2000-08-08")
|
|
||||||
.withGender(Patient.Gender.MALE)
|
|
||||||
.build()
|
|
||||||
)
|
|
||||||
.withConsent(
|
|
||||||
Consent.builder()
|
|
||||||
.withId("1")
|
|
||||||
.withStatus(Consent.Status.ACTIVE)
|
|
||||||
.withPatient("PID")
|
|
||||||
.build()
|
|
||||||
)
|
|
||||||
.withEpisode(
|
|
||||||
Episode.builder()
|
|
||||||
.withId("1")
|
|
||||||
.withPatient("PID")
|
|
||||||
.withPeriod(PeriodStart("2023-08-08"))
|
|
||||||
.build()
|
|
||||||
)
|
|
||||||
.build()
|
|
||||||
|
|
||||||
fun dnpmV2MtbFile(): Mtb {
|
fun dnpmV2MtbFile(): Mtb {
|
||||||
return Mtb().apply {
|
return Mtb().apply {
|
||||||
this.patient = dev.pcvolkmer.mv64e.mtb.Patient().apply {
|
this.patient = dev.pcvolkmer.mv64e.mtb.Patient().apply {
|
||||||
|
@@ -21,8 +21,6 @@ package dev.dnpm.etl.processor.pseudonym
|
|||||||
|
|
||||||
import ca.uhn.fhir.context.FhirContext
|
import ca.uhn.fhir.context.FhirContext
|
||||||
import com.fasterxml.jackson.databind.ObjectMapper
|
import com.fasterxml.jackson.databind.ObjectMapper
|
||||||
import de.ukw.ccc.bwhc.dto.*
|
|
||||||
import de.ukw.ccc.bwhc.dto.Patient
|
|
||||||
import dev.dnpm.etl.processor.config.AppConfigProperties
|
import dev.dnpm.etl.processor.config.AppConfigProperties
|
||||||
import dev.dnpm.etl.processor.config.GIcsConfigProperties
|
import dev.dnpm.etl.processor.config.GIcsConfigProperties
|
||||||
import dev.dnpm.etl.processor.config.JacksonConfig
|
import dev.dnpm.etl.processor.config.JacksonConfig
|
||||||
@@ -51,172 +49,6 @@ class ExtensionsTest {
|
|||||||
return JacksonConfig().objectMapper()
|
return JacksonConfig().objectMapper()
|
||||||
}
|
}
|
||||||
|
|
||||||
@Nested
|
|
||||||
inner class UsingBwhcDatamodel {
|
|
||||||
|
|
||||||
val FAKE_MTB_FILE_PATH = "fake_MTBFile.json"
|
|
||||||
val CLEAN_PATIENT_ID = "5dad2f0b-49c6-47d8-a952-7b9e9e0f7549"
|
|
||||||
|
|
||||||
|
|
||||||
private fun fakeMtbFile(): MtbFile {
|
|
||||||
val mtbFile = ClassPathResource(FAKE_MTB_FILE_PATH).inputStream
|
|
||||||
return getObjectMapper().readValue(mtbFile, MtbFile::class.java)
|
|
||||||
}
|
|
||||||
|
|
||||||
private fun MtbFile.serialized(): String {
|
|
||||||
return getObjectMapper().writeValueAsString(this)
|
|
||||||
}
|
|
||||||
|
|
||||||
@Test
|
|
||||||
fun shouldNotContainCleanPatientId(@Mock pseudonymizeService: PseudonymizeService) {
|
|
||||||
doAnswer {
|
|
||||||
it.arguments[0]
|
|
||||||
"PSEUDO-ID"
|
|
||||||
}.whenever(pseudonymizeService).patientPseudonym(anyValueClass())
|
|
||||||
|
|
||||||
val mtbFile = fakeMtbFile()
|
|
||||||
|
|
||||||
mtbFile.pseudonymizeWith(pseudonymizeService)
|
|
||||||
|
|
||||||
assertThat(mtbFile.patient.id).isEqualTo("PSEUDO-ID")
|
|
||||||
assertThat(mtbFile.serialized()).doesNotContain(CLEAN_PATIENT_ID)
|
|
||||||
}
|
|
||||||
|
|
||||||
@Test
|
|
||||||
fun shouldNotContainAnyUuidAfterRehashingOfIds(@Mock pseudonymizeService: PseudonymizeService) {
|
|
||||||
doAnswer {
|
|
||||||
it.arguments[0]
|
|
||||||
"PSEUDO-ID"
|
|
||||||
}.whenever(pseudonymizeService).patientPseudonym(anyValueClass())
|
|
||||||
|
|
||||||
doAnswer {
|
|
||||||
"TESTDOMAIN"
|
|
||||||
}.whenever(pseudonymizeService).prefix()
|
|
||||||
|
|
||||||
val mtbFile = fakeMtbFile()
|
|
||||||
|
|
||||||
mtbFile.pseudonymizeWith(pseudonymizeService)
|
|
||||||
mtbFile.anonymizeContentWith(pseudonymizeService)
|
|
||||||
|
|
||||||
val pattern =
|
|
||||||
"\"[a-f0-9]{8}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{12}\"".toRegex()
|
|
||||||
.toPattern()
|
|
||||||
val matcher = pattern.matcher(mtbFile.serialized())
|
|
||||||
|
|
||||||
assertThrows<IllegalStateException> {
|
|
||||||
matcher.find()
|
|
||||||
matcher.group()
|
|
||||||
}.also {
|
|
||||||
assertThat(it.message).isEqualTo("No match found")
|
|
||||||
}
|
|
||||||
|
|
||||||
}
|
|
||||||
|
|
||||||
@Test
|
|
||||||
fun shouldRehashIdsWithPrefix(@Mock pseudonymizeService: PseudonymizeService) {
|
|
||||||
doAnswer {
|
|
||||||
it.arguments[0]
|
|
||||||
"PSEUDO-ID"
|
|
||||||
}.whenever(pseudonymizeService).patientPseudonym(anyValueClass())
|
|
||||||
|
|
||||||
doAnswer {
|
|
||||||
"TESTDOMAIN"
|
|
||||||
}.whenever(pseudonymizeService).prefix()
|
|
||||||
|
|
||||||
val mtbFile = MtbFile.builder()
|
|
||||||
.withPatient(
|
|
||||||
Patient.builder()
|
|
||||||
.withId("1")
|
|
||||||
.withBirthDate("2000-08-08")
|
|
||||||
.withGender(Patient.Gender.MALE)
|
|
||||||
.build()
|
|
||||||
)
|
|
||||||
.withConsent(
|
|
||||||
Consent.builder()
|
|
||||||
.withId("1")
|
|
||||||
.withStatus(Consent.Status.ACTIVE)
|
|
||||||
.withPatient("123")
|
|
||||||
.build()
|
|
||||||
)
|
|
||||||
.withEpisode(
|
|
||||||
Episode.builder()
|
|
||||||
.withId("1")
|
|
||||||
.withPatient("1")
|
|
||||||
.withPeriod(PeriodStart("2023-08-08"))
|
|
||||||
.build()
|
|
||||||
)
|
|
||||||
.build()
|
|
||||||
|
|
||||||
mtbFile.pseudonymizeWith(pseudonymizeService)
|
|
||||||
mtbFile.anonymizeContentWith(pseudonymizeService)
|
|
||||||
|
|
||||||
|
|
||||||
assertThat(mtbFile.episode.id)
|
|
||||||
// TESTDOMAIN<sha256(TESTDOMAIN-1)[0-41]>
|
|
||||||
.isEqualTo("TESTDOMAIN44e20a53bbbf9f3ae39626d05df7014dcd77d6098")
|
|
||||||
}
|
|
||||||
|
|
||||||
@Test
|
|
||||||
fun shouldNotThrowExceptionOnNullValues(@Mock pseudonymizeService: PseudonymizeService) {
|
|
||||||
doAnswer {
|
|
||||||
it.arguments[0]
|
|
||||||
"PSEUDO-ID"
|
|
||||||
}.whenever(pseudonymizeService).patientPseudonym(anyValueClass())
|
|
||||||
|
|
||||||
doAnswer {
|
|
||||||
"TESTDOMAIN"
|
|
||||||
}.whenever(pseudonymizeService).prefix()
|
|
||||||
|
|
||||||
val mtbFile = MtbFile.builder()
|
|
||||||
.withPatient(
|
|
||||||
Patient.builder()
|
|
||||||
.withId("1")
|
|
||||||
.withBirthDate("2000-08-08")
|
|
||||||
.withGender(Patient.Gender.MALE)
|
|
||||||
.build()
|
|
||||||
)
|
|
||||||
.withConsent(
|
|
||||||
Consent.builder()
|
|
||||||
.withId("1")
|
|
||||||
.withStatus(Consent.Status.ACTIVE)
|
|
||||||
.withPatient("123")
|
|
||||||
.build()
|
|
||||||
)
|
|
||||||
.withEpisode(
|
|
||||||
Episode.builder()
|
|
||||||
.withId("1")
|
|
||||||
.withPatient("1")
|
|
||||||
.withPeriod(PeriodStart("2023-08-08"))
|
|
||||||
.build()
|
|
||||||
)
|
|
||||||
.withClaims(null)
|
|
||||||
.withDiagnoses(null)
|
|
||||||
.withCarePlans(null)
|
|
||||||
.withClaimResponses(null)
|
|
||||||
.withEcogStatus(null)
|
|
||||||
.withFamilyMemberDiagnoses(null)
|
|
||||||
.withGeneticCounsellingRequests(null)
|
|
||||||
.withHistologyReevaluationRequests(null)
|
|
||||||
.withHistologyReports(null)
|
|
||||||
.withLastGuidelineTherapies(null)
|
|
||||||
.withMolecularPathologyFindings(null)
|
|
||||||
.withMolecularTherapies(null)
|
|
||||||
.withNgsReports(null)
|
|
||||||
.withPreviousGuidelineTherapies(null)
|
|
||||||
.withRebiopsyRequests(null)
|
|
||||||
.withRecommendations(null)
|
|
||||||
.withResponses(null)
|
|
||||||
.withStudyInclusionRequests(null)
|
|
||||||
.withSpecimens(null)
|
|
||||||
.build()
|
|
||||||
|
|
||||||
mtbFile.pseudonymizeWith(pseudonymizeService)
|
|
||||||
mtbFile.anonymizeContentWith(pseudonymizeService)
|
|
||||||
|
|
||||||
assertThat(mtbFile.episode.id).isNotNull()
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
@Nested
|
@Nested
|
||||||
inner class UsingDnpmV2Datamodel {
|
inner class UsingDnpmV2Datamodel {
|
||||||
|
|
||||||
@@ -251,7 +83,7 @@ class ExtensionsTest {
|
|||||||
|
|
||||||
private fun addConsentData(mtbFile: Mtb) {
|
private fun addConsentData(mtbFile: Mtb) {
|
||||||
val gIcsConfigProperties = GIcsConfigProperties("", "", "")
|
val gIcsConfigProperties = GIcsConfigProperties("", "", "")
|
||||||
val appConfigProperties = AppConfigProperties(null, emptyList())
|
val appConfigProperties = AppConfigProperties(emptyList())
|
||||||
|
|
||||||
val bundle = Bundle()
|
val bundle = Bundle()
|
||||||
val dummyConsent = ConsentProcessorTest.getDummyGenomDeConsent()
|
val dummyConsent = ConsentProcessorTest.getDummyGenomDeConsent()
|
||||||
|
@@ -19,8 +19,8 @@
|
|||||||
|
|
||||||
package dev.dnpm.etl.processor.pseudonym
|
package dev.dnpm.etl.processor.pseudonym
|
||||||
|
|
||||||
import de.ukw.ccc.bwhc.dto.*
|
|
||||||
import dev.dnpm.etl.processor.config.PseudonymizeConfigProperties
|
import dev.dnpm.etl.processor.config.PseudonymizeConfigProperties
|
||||||
|
import dev.pcvolkmer.mv64e.mtb.*
|
||||||
import org.assertj.core.api.Assertions.assertThat
|
import org.assertj.core.api.Assertions.assertThat
|
||||||
import org.junit.jupiter.api.Test
|
import org.junit.jupiter.api.Test
|
||||||
import org.junit.jupiter.api.extension.ExtendWith
|
import org.junit.jupiter.api.extension.ExtendWith
|
||||||
@@ -29,31 +29,26 @@ import org.mockito.Mock
|
|||||||
import org.mockito.junit.jupiter.MockitoExtension
|
import org.mockito.junit.jupiter.MockitoExtension
|
||||||
import org.mockito.kotlin.doAnswer
|
import org.mockito.kotlin.doAnswer
|
||||||
import org.mockito.kotlin.whenever
|
import org.mockito.kotlin.whenever
|
||||||
|
import java.time.Instant
|
||||||
|
import java.util.*
|
||||||
|
|
||||||
@ExtendWith(MockitoExtension::class)
|
@ExtendWith(MockitoExtension::class)
|
||||||
class PseudonymizeServiceTest {
|
class PseudonymizeServiceTest {
|
||||||
|
|
||||||
private val mtbFile = MtbFile.builder()
|
private val mtbFile = Mtb.builder()
|
||||||
.withPatient(
|
.patient(
|
||||||
Patient.builder()
|
Patient.builder()
|
||||||
.withId("123")
|
.id("123")
|
||||||
.withBirthDate("2000-08-08")
|
|
||||||
.withGender(Patient.Gender.MALE)
|
|
||||||
.build()
|
.build()
|
||||||
)
|
)
|
||||||
.withConsent(
|
.episodesOfCare(
|
||||||
Consent.builder()
|
listOf(
|
||||||
.withId("1")
|
MtbEpisodeOfCare.builder()
|
||||||
.withStatus(Consent.Status.ACTIVE)
|
.id("1")
|
||||||
.withPatient("123")
|
.patient(Reference.builder().id("123").build())
|
||||||
.build()
|
.period(PeriodDate.builder().start(Date.from(Instant.parse("2021-01-01T00:00:00.00Z"))).build())
|
||||||
)
|
.build()
|
||||||
.withEpisode(
|
)
|
||||||
Episode.builder()
|
|
||||||
.withId("1")
|
|
||||||
.withPatient("123")
|
|
||||||
.withPeriod(PeriodStart("2023-08-08"))
|
|
||||||
.build()
|
|
||||||
)
|
)
|
||||||
.build()
|
.build()
|
||||||
|
|
||||||
@@ -102,4 +97,4 @@ class PseudonymizeServiceTest {
|
|||||||
assertThat(tans.add(tan)).`as`("never the same result!").isTrue
|
assertThat(tans.add(tan)).`as`("never the same result!").isTrue
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
@@ -51,7 +51,7 @@ class ConsentProcessorTest {
|
|||||||
this.objectMapper = jacksonConfig.objectMapper()
|
this.objectMapper = jacksonConfig.objectMapper()
|
||||||
this.fhirContext = JacksonConfig.fhirContext()
|
this.fhirContext = JacksonConfig.fhirContext()
|
||||||
this.gicsConsentService = gicsConsentService
|
this.gicsConsentService = gicsConsentService
|
||||||
this.appConfigProperties = AppConfigProperties(null, emptyList())
|
this.appConfigProperties = AppConfigProperties(emptyList())
|
||||||
this.consentProcessor =
|
this.consentProcessor =
|
||||||
ConsentProcessor(
|
ConsentProcessor(
|
||||||
appConfigProperties,
|
appConfigProperties,
|
||||||
@@ -168,4 +168,4 @@ class ConsentProcessorTest {
|
|||||||
.parseResource<Bundle>(Bundle::class.java, bundle)
|
.parseResource<Bundle>(Bundle::class.java, bundle)
|
||||||
}
|
}
|
||||||
|
|
||||||
}
|
}
|
||||||
|
@@ -20,22 +20,21 @@
|
|||||||
package dev.dnpm.etl.processor.services
|
package dev.dnpm.etl.processor.services
|
||||||
|
|
||||||
import com.fasterxml.jackson.databind.ObjectMapper
|
import com.fasterxml.jackson.databind.ObjectMapper
|
||||||
import de.ukw.ccc.bwhc.dto.*
|
|
||||||
import dev.dnpm.etl.processor.Fingerprint
|
import dev.dnpm.etl.processor.Fingerprint
|
||||||
import dev.dnpm.etl.processor.PatientId
|
import dev.dnpm.etl.processor.PatientId
|
||||||
import dev.dnpm.etl.processor.PatientPseudonym
|
import dev.dnpm.etl.processor.PatientPseudonym
|
||||||
import dev.dnpm.etl.processor.config.AppConfigProperties
|
import dev.dnpm.etl.processor.config.AppConfigProperties
|
||||||
import dev.dnpm.etl.processor.consent.GicsConsentService
|
|
||||||
import dev.dnpm.etl.processor.consent.TtpConsentStatus
|
import dev.dnpm.etl.processor.consent.TtpConsentStatus
|
||||||
import dev.dnpm.etl.processor.monitoring.Request
|
import dev.dnpm.etl.processor.monitoring.Request
|
||||||
import dev.dnpm.etl.processor.monitoring.RequestStatus
|
import dev.dnpm.etl.processor.monitoring.RequestStatus
|
||||||
import dev.dnpm.etl.processor.monitoring.RequestType
|
import dev.dnpm.etl.processor.monitoring.RequestType
|
||||||
import dev.dnpm.etl.processor.output.BwhcV1MtbFileRequest
|
|
||||||
import dev.dnpm.etl.processor.output.DeleteRequest
|
import dev.dnpm.etl.processor.output.DeleteRequest
|
||||||
|
import dev.dnpm.etl.processor.output.DnpmV2MtbFileRequest
|
||||||
import dev.dnpm.etl.processor.output.MtbFileSender
|
import dev.dnpm.etl.processor.output.MtbFileSender
|
||||||
import dev.dnpm.etl.processor.output.RestMtbFileSender
|
import dev.dnpm.etl.processor.output.RestMtbFileSender
|
||||||
import dev.dnpm.etl.processor.pseudonym.PseudonymizeService
|
import dev.dnpm.etl.processor.pseudonym.PseudonymizeService
|
||||||
import dev.dnpm.etl.processor.randomRequestId
|
import dev.dnpm.etl.processor.randomRequestId
|
||||||
|
import dev.pcvolkmer.mv64e.mtb.*
|
||||||
import org.assertj.core.api.Assertions.assertThat
|
import org.assertj.core.api.Assertions.assertThat
|
||||||
import org.junit.jupiter.api.BeforeEach
|
import org.junit.jupiter.api.BeforeEach
|
||||||
import org.junit.jupiter.api.Test
|
import org.junit.jupiter.api.Test
|
||||||
@@ -49,6 +48,7 @@ import org.mockito.kotlin.argumentCaptor
|
|||||||
import org.mockito.kotlin.whenever
|
import org.mockito.kotlin.whenever
|
||||||
import org.springframework.context.ApplicationEventPublisher
|
import org.springframework.context.ApplicationEventPublisher
|
||||||
import java.time.Instant
|
import java.time.Instant
|
||||||
|
import java.util.*
|
||||||
|
|
||||||
|
|
||||||
@ExtendWith(MockitoExtension::class)
|
@ExtendWith(MockitoExtension::class)
|
||||||
@@ -77,7 +77,7 @@ class RequestProcessorTest {
|
|||||||
this.sender = sender
|
this.sender = sender
|
||||||
this.requestService = requestService
|
this.requestService = requestService
|
||||||
this.applicationEventPublisher = applicationEventPublisher
|
this.applicationEventPublisher = applicationEventPublisher
|
||||||
this.appConfigProperties = AppConfigProperties(null)
|
this.appConfigProperties = AppConfigProperties()
|
||||||
this.consentProcessor = consentProcessor
|
this.consentProcessor = consentProcessor
|
||||||
|
|
||||||
val objectMapper = ObjectMapper()
|
val objectMapper = ObjectMapper()
|
||||||
@@ -102,7 +102,7 @@ class RequestProcessorTest {
|
|||||||
randomRequestId(),
|
randomRequestId(),
|
||||||
PatientPseudonym("TEST_12345678901"),
|
PatientPseudonym("TEST_12345678901"),
|
||||||
PatientId("P1"),
|
PatientId("P1"),
|
||||||
Fingerprint("zdlzv5s5ydmd4ktw2v5piohegc4jcyrm6j66bq6tv2uxuerndmga"),
|
Fingerprint("6vkiti5bk6ikwifpajpt7cygmd3dvw54d6lwfhzlynb3pqtzferq"),
|
||||||
RequestType.MTB_FILE,
|
RequestType.MTB_FILE,
|
||||||
RequestStatus.SUCCESS,
|
RequestStatus.SUCCESS,
|
||||||
Instant.parse("2023-08-08T02:00:00Z")
|
Instant.parse("2023-08-08T02:00:00Z")
|
||||||
@@ -119,29 +119,24 @@ class RequestProcessorTest {
|
|||||||
|
|
||||||
doAnswer {
|
doAnswer {
|
||||||
it.arguments[0]
|
it.arguments[0]
|
||||||
}.whenever(transformationService).transform(any<MtbFile>())
|
}.whenever(transformationService).transform(any<Mtb>())
|
||||||
|
|
||||||
val mtbFile = MtbFile.builder()
|
whenever(consentProcessor.consentGatedCheckAndTryEmbedding(any())).thenReturn(true)
|
||||||
.withPatient(
|
|
||||||
|
val mtbFile = Mtb.builder()
|
||||||
|
.patient(
|
||||||
Patient.builder()
|
Patient.builder()
|
||||||
.withId("1")
|
.id("123")
|
||||||
.withBirthDate("2000-08-08")
|
|
||||||
.withGender(Patient.Gender.MALE)
|
|
||||||
.build()
|
.build()
|
||||||
)
|
)
|
||||||
.withConsent(
|
.episodesOfCare(
|
||||||
Consent.builder()
|
listOf(
|
||||||
.withId("1")
|
MtbEpisodeOfCare.builder()
|
||||||
.withStatus(Consent.Status.ACTIVE)
|
.id("1")
|
||||||
.withPatient("123")
|
.patient(Reference.builder().id("123").build())
|
||||||
.build()
|
.period(PeriodDate.builder().start(Date.from(Instant.parse("2023-08-08T02:00:00.00Z"))).build())
|
||||||
)
|
.build()
|
||||||
.withEpisode(
|
)
|
||||||
Episode.builder()
|
|
||||||
.withId("1")
|
|
||||||
.withPatient("1")
|
|
||||||
.withPeriod(PeriodStart("2023-08-08"))
|
|
||||||
.build()
|
|
||||||
)
|
)
|
||||||
.build()
|
.build()
|
||||||
|
|
||||||
@@ -161,7 +156,7 @@ class RequestProcessorTest {
|
|||||||
randomRequestId(),
|
randomRequestId(),
|
||||||
PatientPseudonym("TEST_12345678901"),
|
PatientPseudonym("TEST_12345678901"),
|
||||||
PatientId("P1"),
|
PatientId("P1"),
|
||||||
Fingerprint("zdlzv5s5ydmd4ktw2v5piohegc4jcyrm6j66bq6tv2uxuerndmga"),
|
Fingerprint("4gcjwtjjtcczybsljxepdfpkaeusvd7g3vogfqpmphyffyzfx7dq"),
|
||||||
RequestType.MTB_FILE,
|
RequestType.MTB_FILE,
|
||||||
RequestStatus.SUCCESS,
|
RequestStatus.SUCCESS,
|
||||||
Instant.parse("2023-08-08T02:00:00Z")
|
Instant.parse("2023-08-08T02:00:00Z")
|
||||||
@@ -178,29 +173,24 @@ class RequestProcessorTest {
|
|||||||
|
|
||||||
doAnswer {
|
doAnswer {
|
||||||
it.arguments[0]
|
it.arguments[0]
|
||||||
}.whenever(transformationService).transform(any<MtbFile>())
|
}.whenever(transformationService).transform(any<Mtb>())
|
||||||
|
|
||||||
val mtbFile = MtbFile.builder()
|
whenever(consentProcessor.consentGatedCheckAndTryEmbedding(any())).thenReturn(true)
|
||||||
.withPatient(
|
|
||||||
|
val mtbFile = Mtb.builder()
|
||||||
|
.patient(
|
||||||
Patient.builder()
|
Patient.builder()
|
||||||
.withId("1")
|
.id("123")
|
||||||
.withBirthDate("2000-08-08")
|
|
||||||
.withGender(Patient.Gender.MALE)
|
|
||||||
.build()
|
.build()
|
||||||
)
|
)
|
||||||
.withConsent(
|
.episodesOfCare(
|
||||||
Consent.builder()
|
listOf(
|
||||||
.withId("1")
|
MtbEpisodeOfCare.builder()
|
||||||
.withStatus(Consent.Status.ACTIVE)
|
.id("1")
|
||||||
.withPatient("123")
|
.patient(Reference.builder().id("123").build())
|
||||||
.build()
|
.period(PeriodDate.builder().start(Date.from(Instant.parse("2021-01-01T00:00:00.00Z"))).build())
|
||||||
)
|
.build()
|
||||||
.withEpisode(
|
)
|
||||||
Episode.builder()
|
|
||||||
.withId("1")
|
|
||||||
.withPatient("1")
|
|
||||||
.withPeriod(PeriodStart("2023-08-08"))
|
|
||||||
.build()
|
|
||||||
)
|
)
|
||||||
.build()
|
.build()
|
||||||
|
|
||||||
@@ -233,7 +223,7 @@ class RequestProcessorTest {
|
|||||||
|
|
||||||
doAnswer {
|
doAnswer {
|
||||||
MtbFileSender.Response(status = RequestStatus.SUCCESS)
|
MtbFileSender.Response(status = RequestStatus.SUCCESS)
|
||||||
}.whenever(sender).send(any<BwhcV1MtbFileRequest>())
|
}.whenever(sender).send(any<DnpmV2MtbFileRequest>())
|
||||||
|
|
||||||
doAnswer {
|
doAnswer {
|
||||||
it.arguments[0] as String
|
it.arguments[0] as String
|
||||||
@@ -241,29 +231,24 @@ class RequestProcessorTest {
|
|||||||
|
|
||||||
doAnswer {
|
doAnswer {
|
||||||
it.arguments[0]
|
it.arguments[0]
|
||||||
}.whenever(transformationService).transform(any<MtbFile>())
|
}.whenever(transformationService).transform(any<Mtb>())
|
||||||
|
|
||||||
val mtbFile = MtbFile.builder()
|
whenever(consentProcessor.consentGatedCheckAndTryEmbedding(any())).thenReturn(true)
|
||||||
.withPatient(
|
|
||||||
|
val mtbFile = Mtb.builder()
|
||||||
|
.patient(
|
||||||
Patient.builder()
|
Patient.builder()
|
||||||
.withId("1")
|
.id("123")
|
||||||
.withBirthDate("2000-08-08")
|
|
||||||
.withGender(Patient.Gender.MALE)
|
|
||||||
.build()
|
.build()
|
||||||
)
|
)
|
||||||
.withConsent(
|
.episodesOfCare(
|
||||||
Consent.builder()
|
listOf(
|
||||||
.withId("1")
|
MtbEpisodeOfCare.builder()
|
||||||
.withStatus(Consent.Status.ACTIVE)
|
.id("1")
|
||||||
.withPatient("123")
|
.patient(Reference.builder().id("123").build())
|
||||||
.build()
|
.period(PeriodDate.builder().start(Date.from(Instant.parse("2021-01-01T00:00:00.00Z"))).build())
|
||||||
)
|
.build()
|
||||||
.withEpisode(
|
)
|
||||||
Episode.builder()
|
|
||||||
.withId("1")
|
|
||||||
.withPatient("1")
|
|
||||||
.withPeriod(PeriodStart("2023-08-08"))
|
|
||||||
.build()
|
|
||||||
)
|
)
|
||||||
.build()
|
.build()
|
||||||
|
|
||||||
@@ -296,7 +281,7 @@ class RequestProcessorTest {
|
|||||||
|
|
||||||
doAnswer {
|
doAnswer {
|
||||||
MtbFileSender.Response(status = RequestStatus.ERROR)
|
MtbFileSender.Response(status = RequestStatus.ERROR)
|
||||||
}.whenever(sender).send(any<BwhcV1MtbFileRequest>())
|
}.whenever(sender).send(any<DnpmV2MtbFileRequest>())
|
||||||
|
|
||||||
doAnswer {
|
doAnswer {
|
||||||
it.arguments[0] as String
|
it.arguments[0] as String
|
||||||
@@ -304,29 +289,36 @@ class RequestProcessorTest {
|
|||||||
|
|
||||||
doAnswer {
|
doAnswer {
|
||||||
it.arguments[0]
|
it.arguments[0]
|
||||||
}.whenever(transformationService).transform(any<MtbFile>())
|
}.whenever(transformationService).transform(any<Mtb>())
|
||||||
|
|
||||||
val mtbFile = MtbFile.builder()
|
whenever(consentProcessor.consentGatedCheckAndTryEmbedding(any())).thenReturn(true)
|
||||||
.withPatient(
|
|
||||||
|
val mtbFile = Mtb.builder()
|
||||||
|
.patient(
|
||||||
Patient.builder()
|
Patient.builder()
|
||||||
.withId("1")
|
.id("123")
|
||||||
.withBirthDate("2000-08-08")
|
|
||||||
.withGender(Patient.Gender.MALE)
|
|
||||||
.build()
|
.build()
|
||||||
)
|
)
|
||||||
.withConsent(
|
.metadata(
|
||||||
Consent.builder()
|
MvhMetadata
|
||||||
.withId("1")
|
.builder()
|
||||||
.withStatus(Consent.Status.ACTIVE)
|
.modelProjectConsent(
|
||||||
.withPatient("123")
|
ModelProjectConsent
|
||||||
|
.builder()
|
||||||
|
.provisions(
|
||||||
|
listOf(Provision.builder().type(ConsentProvision.PERMIT).purpose(ModelProjectConsentPurpose.SEQUENCING).build())
|
||||||
|
).build()
|
||||||
|
)
|
||||||
.build()
|
.build()
|
||||||
)
|
)
|
||||||
.withEpisode(
|
.episodesOfCare(
|
||||||
Episode.builder()
|
listOf(
|
||||||
.withId("1")
|
MtbEpisodeOfCare.builder()
|
||||||
.withPatient("1")
|
.id("1")
|
||||||
.withPeriod(PeriodStart("2023-08-08"))
|
.patient(Reference.builder().id("123").build())
|
||||||
.build()
|
.period(PeriodDate.builder().start(Date.from(Instant.parse("2021-01-01T00:00:00.00Z"))).build())
|
||||||
|
.build()
|
||||||
|
)
|
||||||
)
|
)
|
||||||
.build()
|
.build()
|
||||||
|
|
||||||
@@ -426,33 +418,28 @@ class RequestProcessorTest {
|
|||||||
|
|
||||||
doAnswer {
|
doAnswer {
|
||||||
it.arguments[0]
|
it.arguments[0]
|
||||||
}.whenever(transformationService).transform(any<MtbFile>())
|
}.whenever(transformationService).transform(any<Mtb>())
|
||||||
|
|
||||||
doAnswer {
|
doAnswer {
|
||||||
MtbFileSender.Response(status = RequestStatus.SUCCESS)
|
MtbFileSender.Response(status = RequestStatus.SUCCESS)
|
||||||
}.whenever(sender).send(any<BwhcV1MtbFileRequest>())
|
}.whenever(sender).send(any<DnpmV2MtbFileRequest>())
|
||||||
|
|
||||||
val mtbFile = MtbFile.builder()
|
whenever(consentProcessor.consentGatedCheckAndTryEmbedding(any())).thenReturn(true)
|
||||||
.withPatient(
|
|
||||||
|
val mtbFile = Mtb.builder()
|
||||||
|
.patient(
|
||||||
Patient.builder()
|
Patient.builder()
|
||||||
.withId("1")
|
.id("123")
|
||||||
.withBirthDate("2000-08-08")
|
|
||||||
.withGender(Patient.Gender.MALE)
|
|
||||||
.build()
|
.build()
|
||||||
)
|
)
|
||||||
.withConsent(
|
.episodesOfCare(
|
||||||
Consent.builder()
|
listOf(
|
||||||
.withId("1")
|
MtbEpisodeOfCare.builder()
|
||||||
.withStatus(Consent.Status.ACTIVE)
|
.id("1")
|
||||||
.withPatient("123")
|
.patient(Reference.builder().id("123").build())
|
||||||
.build()
|
.period(PeriodDate.builder().start(Date.from(Instant.parse("2021-01-01T00:00:00.00Z"))).build())
|
||||||
)
|
.build()
|
||||||
.withEpisode(
|
)
|
||||||
Episode.builder()
|
|
||||||
.withId("1")
|
|
||||||
.withPatient("1")
|
|
||||||
.withPeriod(PeriodStart("2023-08-08"))
|
|
||||||
.build()
|
|
||||||
)
|
)
|
||||||
.build()
|
.build()
|
||||||
|
|
||||||
|
@@ -19,20 +19,11 @@
|
|||||||
|
|
||||||
package dev.dnpm.etl.processor.services
|
package dev.dnpm.etl.processor.services
|
||||||
|
|
||||||
import de.ukw.ccc.bwhc.dto.Consent
|
|
||||||
import de.ukw.ccc.bwhc.dto.Diagnosis
|
|
||||||
import de.ukw.ccc.bwhc.dto.Icd10
|
|
||||||
import de.ukw.ccc.bwhc.dto.MtbFile
|
|
||||||
import dev.dnpm.etl.processor.config.JacksonConfig
|
import dev.dnpm.etl.processor.config.JacksonConfig
|
||||||
import dev.pcvolkmer.mv64e.mtb.ConsentProvision
|
import dev.pcvolkmer.mv64e.mtb.*
|
||||||
import dev.pcvolkmer.mv64e.mtb.ModelProjectConsent
|
|
||||||
import dev.pcvolkmer.mv64e.mtb.ModelProjectConsentPurpose
|
|
||||||
import org.assertj.core.api.Assertions.assertThat
|
import org.assertj.core.api.Assertions.assertThat
|
||||||
import org.junit.jupiter.api.BeforeEach
|
import org.junit.jupiter.api.BeforeEach
|
||||||
import org.junit.jupiter.api.Test
|
import org.junit.jupiter.api.Test
|
||||||
import dev.pcvolkmer.mv64e.mtb.Mtb
|
|
||||||
import dev.pcvolkmer.mv64e.mtb.MvhMetadata
|
|
||||||
import dev.pcvolkmer.mv64e.mtb.Provision
|
|
||||||
import org.hl7.fhir.instance.model.api.IBaseResource
|
import org.hl7.fhir.instance.model.api.IBaseResource
|
||||||
import java.time.Instant
|
import java.time.Instant
|
||||||
import java.util.Date
|
import java.util.Date
|
||||||
@@ -45,82 +36,59 @@ class TransformationServiceTest {
|
|||||||
fun setup() {
|
fun setup() {
|
||||||
this.service = TransformationService(
|
this.service = TransformationService(
|
||||||
JacksonConfig().objectMapper(), listOf(
|
JacksonConfig().objectMapper(), listOf(
|
||||||
Transformation.of("consent.status") from Consent.Status.ACTIVE to Consent.Status.REJECTED,
|
Transformation.of("diagnoses[*].code.version") from "2013" to "2014",
|
||||||
Transformation.of("diagnoses[*].icd10.version") from "2013" to "2014",
|
|
||||||
)
|
)
|
||||||
)
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
@Test
|
@Test
|
||||||
fun shouldTransformMtbFile() {
|
fun shouldTransformMtbFile() {
|
||||||
val mtbFile = MtbFile.builder().withDiagnoses(
|
val mtbFile = Mtb.builder().diagnoses(
|
||||||
listOf(
|
listOf(
|
||||||
Diagnosis.builder().withId("1234").withIcd10(Icd10("F79.9").also {
|
MtbDiagnosis.builder().id("1234").code(Coding.builder().code("F79.9").version("2013").build()).build()
|
||||||
it.version = "2013"
|
|
||||||
}).build()
|
|
||||||
)
|
)
|
||||||
).build()
|
).build()
|
||||||
|
|
||||||
val actual = this.service.transform(mtbFile)
|
val actual = this.service.transform(mtbFile)
|
||||||
|
|
||||||
assertThat(actual).isNotNull
|
assertThat(actual).isNotNull
|
||||||
assertThat(actual.diagnoses[0].icd10.version).isEqualTo("2014")
|
assertThat(actual.diagnoses[0].code.version).isEqualTo("2014")
|
||||||
}
|
}
|
||||||
|
|
||||||
@Test
|
@Test
|
||||||
fun shouldOnlyTransformGivenValues() {
|
fun shouldOnlyTransformGivenValues() {
|
||||||
val mtbFile = MtbFile.builder().withDiagnoses(
|
val mtbFile = Mtb.builder().diagnoses(
|
||||||
listOf(
|
listOf(
|
||||||
Diagnosis.builder().withId("1234").withIcd10(Icd10("F79.9").also {
|
MtbDiagnosis.builder().id("1234").code(Coding.builder().code("F79.9").version("2013").build()).build(),
|
||||||
it.version = "2013"
|
MtbDiagnosis.builder().id("1234").code(Coding.builder().code("F79.8").version("2019").build()).build()
|
||||||
}).build(),
|
|
||||||
Diagnosis.builder().withId("5678").withIcd10(Icd10("F79.8").also {
|
|
||||||
it.version = "2019"
|
|
||||||
}).build()
|
|
||||||
)
|
)
|
||||||
).build()
|
).build()
|
||||||
|
|
||||||
val actual = this.service.transform(mtbFile)
|
val actual = this.service.transform(mtbFile)
|
||||||
|
|
||||||
assertThat(actual).isNotNull
|
assertThat(actual).isNotNull
|
||||||
assertThat(actual.diagnoses[0].icd10.code).isEqualTo("F79.9")
|
assertThat(actual.diagnoses[0].code.code).isEqualTo("F79.9")
|
||||||
assertThat(actual.diagnoses[0].icd10.version).isEqualTo("2014")
|
assertThat(actual.diagnoses[0].code.version).isEqualTo("2014")
|
||||||
assertThat(actual.diagnoses[1].icd10.code).isEqualTo("F79.8")
|
assertThat(actual.diagnoses[1].code.code).isEqualTo("F79.8")
|
||||||
assertThat(actual.diagnoses[1].icd10.version).isEqualTo("2019")
|
assertThat(actual.diagnoses[1].code.version).isEqualTo("2019")
|
||||||
}
|
|
||||||
|
|
||||||
@Test
|
|
||||||
fun shouldTransformMtbFileWithConsentEnum() {
|
|
||||||
val mtbFile = MtbFile.builder().withConsent(
|
|
||||||
Consent("123", "456", Consent.Status.ACTIVE)
|
|
||||||
).build()
|
|
||||||
|
|
||||||
val actual = this.service.transform(mtbFile)
|
|
||||||
|
|
||||||
assertThat(actual.consent).isNotNull
|
|
||||||
assertThat(actual.consent.status).isEqualTo(Consent.Status.REJECTED)
|
|
||||||
}
|
}
|
||||||
|
|
||||||
@Test
|
@Test
|
||||||
fun shouldTransformConsentValues() {
|
fun shouldTransformConsentValues() {
|
||||||
val mtbFile = MtbFile.builder().withDiagnoses(
|
val mtbFile = Mtb.builder().diagnoses(
|
||||||
listOf(
|
listOf(
|
||||||
Diagnosis.builder().withId("1234").withIcd10(Icd10("F79.9").also {
|
MtbDiagnosis.builder().id("1234").code(Coding.builder().code("F79.9").version("2013").build()).build(),
|
||||||
it.version = "2013"
|
MtbDiagnosis.builder().id("1234").code(Coding.builder().code("F79.8").version("2019").build()).build()
|
||||||
}).build(),
|
|
||||||
Diagnosis.builder().withId("5678").withIcd10(Icd10("F79.8").also {
|
|
||||||
it.version = "2019"
|
|
||||||
}).build()
|
|
||||||
)
|
)
|
||||||
).build()
|
).build()
|
||||||
|
|
||||||
val actual = this.service.transform(mtbFile)
|
val actual = this.service.transform(mtbFile)
|
||||||
|
|
||||||
assertThat(actual).isNotNull
|
assertThat(actual).isNotNull
|
||||||
assertThat(actual.diagnoses[0].icd10.code).isEqualTo("F79.9")
|
assertThat(actual.diagnoses[0].code.code).isEqualTo("F79.9")
|
||||||
assertThat(actual.diagnoses[0].icd10.version).isEqualTo("2014")
|
assertThat(actual.diagnoses[0].code.version).isEqualTo("2014")
|
||||||
assertThat(actual.diagnoses[1].icd10.code).isEqualTo("F79.8")
|
assertThat(actual.diagnoses[1].code.code).isEqualTo("F79.8")
|
||||||
assertThat(actual.diagnoses[1].icd10.version).isEqualTo("2019")
|
assertThat(actual.diagnoses[1].code.version).isEqualTo("2019")
|
||||||
}
|
}
|
||||||
|
|
||||||
@Test
|
@Test
|
||||||
@@ -155,5 +123,4 @@ class TransformationServiceTest {
|
|||||||
|
|
||||||
}
|
}
|
||||||
|
|
||||||
|
}
|
||||||
}
|
|
||||||
|
Reference in New Issue
Block a user