1
0
mirror of https://github.com/pcvolkmer/etl-processor.git synced 2025-04-19 17:26:51 +00:00

276 Commits

Author SHA1 Message Date
72295202ec Code cleanup 2023-08-12 22:19:29 +02:00
bc48a7217e Add more information about usage in an ETl process 2023-08-11 14:37:48 +02:00
0e1034d964 New version and add status badge to README.md v0.1.1 2023-08-11 09:47:20 +02:00
6ecb439007
Issue #3: Detect the request type of request with last known status (#5) 2023-08-11 09:22:54 +02:00
cb9c590472
Issue #2: Do not serialize JSON string as custom string (#4)
In addition to that, if REST request did not contain a response body, use empty
string as data quality report string.
2023-08-11 09:13:45 +02:00
044d01534b Build and deploy docker image on new release v0.1.0 2023-08-10 20:29:43 +02:00
4dde13e79a Run tests on each tag 2023-08-10 18:59:04 +02:00
55153d8050 Add information about docker image 2023-08-10 18:55:33 +02:00
25ec557c25 Change condition when to run docker job 2023-08-10 18:44:33 +02:00
2264d85bd1 Run docker workflow after test workflow 2023-08-10 18:29:25 +02:00
2ec5e27a40 Full setup for docker build 2023-08-10 18:07:25 +02:00
d24d9a7fd0 Add docker deploy workflow 2023-08-10 17:46:11 +02:00
1e29ecc891 Fix event name in workflow file 2023-08-10 17:32:02 +02:00
b956eba6c7 Add workflow to run tests on push or pull request 2023-08-10 17:27:14 +02:00
002b0618cf Add tests for KafkaMtbFileSender 2023-08-10 13:35:35 +02:00
2b42a4d262 Tests for RestMtbFileSender 2023-08-10 12:14:13 +02:00
501762d451 Add test logging 2023-08-09 18:32:03 +02:00
1a640ff9df Decouple request and response processing 2023-08-09 18:15:20 +02:00
7f048e2483 Do not append custom prefix to gPAS pseudonym 2023-08-09 12:26:57 +02:00
47830ed9f7 Use single MtbFileSender 2023-08-09 10:34:23 +02:00
13bfa0018d Change endpoint configuration to select single endpoint
* If REST endpoint is configured, it will be used
* If Kafka endpoint is configured, it will be used
* If both endpoints are configured, REST configuration has precedence and will be used
2023-08-09 10:22:03 +02:00
7739afad1f Handle MTB File with rejected consent as deletion request 2023-08-09 08:13:27 +02:00
6ad6ee13a1 Ignore unknown properties in DataQualityResponse 2023-08-08 19:20:45 +02:00
536ecbbd56 Add tests for error response status 2023-08-08 18:52:03 +02:00
422441a3b3 Add tests for RequestProcessor 2023-08-08 16:46:02 +02:00
b75328b74d Move integration tests into own source-set 2023-08-08 15:16:58 +02:00
4051b5094c Keep database testcontainer alive until all tests are done 2023-08-08 14:58:10 +02:00
bcc23f6b14 Add RequestService to handle access to requests 2023-08-08 14:57:31 +02:00
1fc09d691e Rename test class to match applications main class name 2023-08-08 13:34:53 +02:00
3039b4b2a7 Add basic Testcontainers test setup 2023-08-08 13:23:37 +02:00
3dea664999 Update Spring Boot dependencies 2023-08-08 12:46:04 +02:00
459ad59c1d Do not detect duplicates after deletion request 2023-08-04 11:43:23 +02:00
b14f2c1794 Add information about 'no connection' responses 2023-08-03 16:18:16 +02:00
ec76c775d9 Explicit producer topic configuration 2023-08-03 16:04:57 +02:00
3dcee41569 Implement delete request using Apache Kafka
This is implemented using a fake MTB file containing a rejected
consent state and will be mapped to HTTP DELETE on kafka-to-bwhc
consumer.
2023-08-03 15:14:49 +02:00
ac91620651 Use Map as status body since it contains JSON 2023-08-03 12:59:53 +02:00
577509e6f2 Map 'status_code' and 'status code' to same data value 2023-08-02 16:53:09 +02:00
7f8b21efd2 Handle not parsable data quality reports 2023-08-02 16:23:16 +02:00
70d4fa2f0f Use duplication fingerprinting based on MTB file requests only 2023-08-02 16:10:53 +02:00
35cb258b13 Do not return specific status code based on remote status code 2023-08-02 16:10:18 +02:00
Jakub Lidke
a075f73162 feat: add Dockerfile for build within docker environment and run application within a container. 2023-08-02 15:19:38 +02:00
51cf7a7917 Add processor to handle responses from Kafka topic 2023-08-02 12:00:04 +02:00
db631fbd8e Added more information about pseudonym generators 2023-08-01 16:06:57 +02:00
857bd157a0 Modified config property to select pseudonym generator 2023-08-01 16:06:40 +02:00
ccf6908223 Added statistics update using SSE for delete requests 2023-08-01 13:54:26 +02:00
7a7ea8c96d Extract request processing 2023-08-01 12:39:15 +02:00
11ed3a6296 Show request type in addition to state and ID 2023-08-01 08:53:43 +02:00
9c9760c3ec Add delete requests to be shown in statistics page 2023-08-01 08:46:00 +02:00
4f7f5e4d89 Add request type to be saved with each request 2023-08-01 08:39:30 +02:00
a1e56f1596 Handle delete requests by using REST request to bwHC
Kafka implementation is not implemented yet.
2023-07-31 19:25:15 +02:00