1
0
mirror of https://github.com/pcvolkmer/etl-processor.git synced 2025-04-20 01:36:50 +00:00

120 Commits

Author SHA1 Message Date
ced6609d9a fix: add info severity to data quality report 2023-12-05 14:24:40 +01:00
3e45de56cf feat: add page that shows transformation configuration 2023-12-04 17:35:44 +01:00
effcdd811f style: add colored table rows for requests 2023-12-04 16:11:02 +01:00
eb24995ed9 Issue #12: Log transformation count applied on application start 2023-10-05 12:35:29 +02:00
4196664060 Issue #12: Transform MTBFile objects by using transformation rules 2023-10-05 12:09:56 +02:00
1e1db1c4d9 Issue #12: Add application config for transformation configuration 2023-10-05 11:37:10 +02:00
7440fe1e23 Issue #12: Basic implementation of transformation service 2023-10-05 10:51:49 +02:00
jlidke
bf8f87b261
fix: removed gaps system from GPAS pseudonym value. Also added clean up method, which will replace filename invalid characters witch '_'. (#11) 2023-09-04 15:41:22 +02:00
79709caa39 Merge remote-tracking branch 'origin/add-docker-build' 2023-08-30 13:29:06 +02:00
Jakub Lidke
edafe30a4b chore: added log msg to GpasPseudonymGenerator 2023-08-30 11:51:08 +02:00
5e93e834ad Remove comment to use host alias 2023-08-30 10:24:48 +02:00
Jakub Lidke
5e5bd579fb test: * added additional external host 'localhost', now we can connect without additional host alias. * added akhq to dev-compose.yml 2023-08-30 10:21:38 +02:00
Jakub Lidke
532254593f test: * added additional external host 'localhost', now we can connect without additional host alias. * added akhq to dev-compose.yml 2023-08-28 12:47:09 +02:00
Jakub Lidke
aa40da4995 chore: dev kafka is available via localhost, now. 2023-08-25 13:11:10 +02:00
9921e1e684 Throw PseudonymRequestFailed exception with error message
This will throw an exception with error message describing what the error is instead of
having a more generic NoSuchElementException to be thrown if Optional.get() has no value
after calling findFirst() on an empty stream.
2023-08-19 11:45:21 +02:00
8dc82225a4
Issue #7: Send and expect requestId in record body, not in record key (#8) 2023-08-16 15:25:46 +02:00
2eb5cc61b9 Change Kafka response body JSON alias 2023-08-15 10:58:17 +02:00
66dc96680d Update dev config and added related information into README.md 2023-08-15 01:09:22 +02:00
72295202ec Code cleanup 2023-08-12 22:19:29 +02:00
6ecb439007
Issue #3: Detect the request type of request with last known status (#5) 2023-08-11 09:22:54 +02:00
cb9c590472
Issue #2: Do not serialize JSON string as custom string (#4)
In addition to that, if REST request did not contain a response body, use empty
string as data quality report string.
2023-08-11 09:13:45 +02:00
002b0618cf Add tests for KafkaMtbFileSender 2023-08-10 13:35:35 +02:00
2b42a4d262 Tests for RestMtbFileSender 2023-08-10 12:14:13 +02:00
1a640ff9df Decouple request and response processing 2023-08-09 18:15:20 +02:00
7f048e2483 Do not append custom prefix to gPAS pseudonym 2023-08-09 12:26:57 +02:00
47830ed9f7 Use single MtbFileSender 2023-08-09 10:34:23 +02:00
13bfa0018d Change endpoint configuration to select single endpoint
* If REST endpoint is configured, it will be used
* If Kafka endpoint is configured, it will be used
* If both endpoints are configured, REST configuration has precedence and will be used
2023-08-09 10:22:03 +02:00
7739afad1f Handle MTB File with rejected consent as deletion request 2023-08-09 08:13:27 +02:00
6ad6ee13a1 Ignore unknown properties in DataQualityResponse 2023-08-08 19:20:45 +02:00
422441a3b3 Add tests for RequestProcessor 2023-08-08 16:46:02 +02:00
bcc23f6b14 Add RequestService to handle access to requests 2023-08-08 14:57:31 +02:00
459ad59c1d Do not detect duplicates after deletion request 2023-08-04 11:43:23 +02:00
b14f2c1794 Add information about 'no connection' responses 2023-08-03 16:18:16 +02:00
ec76c775d9 Explicit producer topic configuration 2023-08-03 16:04:57 +02:00
3dcee41569 Implement delete request using Apache Kafka
This is implemented using a fake MTB file containing a rejected
consent state and will be mapped to HTTP DELETE on kafka-to-bwhc
consumer.
2023-08-03 15:14:49 +02:00
ac91620651 Use Map as status body since it contains JSON 2023-08-03 12:59:53 +02:00
577509e6f2 Map 'status_code' and 'status code' to same data value 2023-08-02 16:53:09 +02:00
7f8b21efd2 Handle not parsable data quality reports 2023-08-02 16:23:16 +02:00
70d4fa2f0f Use duplication fingerprinting based on MTB file requests only 2023-08-02 16:10:53 +02:00
35cb258b13 Do not return specific status code based on remote status code 2023-08-02 16:10:18 +02:00
51cf7a7917 Add processor to handle responses from Kafka topic 2023-08-02 12:00:04 +02:00
857bd157a0 Modified config property to select pseudonym generator 2023-08-01 16:06:40 +02:00
ccf6908223 Added statistics update using SSE for delete requests 2023-08-01 13:54:26 +02:00
7a7ea8c96d Extract request processing 2023-08-01 12:39:15 +02:00
11ed3a6296 Show request type in addition to state and ID 2023-08-01 08:53:43 +02:00
9c9760c3ec Add delete requests to be shown in statistics page 2023-08-01 08:46:00 +02:00
4f7f5e4d89 Add request type to be saved with each request 2023-08-01 08:39:30 +02:00
a1e56f1596 Handle delete requests by using REST request to bwHC
Kafka implementation is not implemented yet.
2023-07-31 19:25:15 +02:00
e1d05ef574 Mark delete requests with errors or success as expected 2023-07-31 19:23:12 +02:00
9af0e159b0 Enable patient data deletion 2023-07-31 17:08:44 +02:00