Evaluator(s)
Michał Kozak (PSNC)
Evaluation points
The main goal of this evaluation was to test the performance of the education portal search function. The test was executed on the PSNC Hadoop cluster and WCPT medical dataset.
Assessment of measurable points
The mean query time is the mean of 10 iterations.
MDC handled one request at a time.
| Metric | Description | Metric baseline | Metric goal | July 24, 2014 | July 29, 2014 | |
|---|---|---|---|---|---|---|
| MeanQueryTimeInSeconds | medical cases by ICD10 code | - | - | 2.095 |
2.712 | |
| MeanQueryTimeInSeconds | medical cases by ICD9 code | - | - | 9.345 |
12.660 | |
| MeanQueryTimeInSeconds | medical cases by patient's city | - | - | 7.247 |
8.360 |
|
| MeanQueryTimeInSeconds | medical cases by patient's sex | - | - | 13.809 |
14.814 |
|
| MeanQueryTimeInSeconds | medical cases by patient's age | - | - | 5.320 |
5.734 |
|
| MeanQueryTimeInSeconds | medical cases by visit's dates | - | - | 5.160 |
6.296 |
|
| MeanQueryTimeInSeconds | medical cases by laboratory tests | - | - | 3.996 |
4.123 |
|
| MeanQueryTimeInSeconds | medical cases by all of the above criteria | - | - | 10.023 |
10.980 |
MDC handled ten request at a time.
| Metric | Description | Metric baseline | Metric goal | July 24, 2014 | July 29, 2014 |
|
|---|---|---|---|---|---|---|
| MeanQueryTimeInSeconds | medical cases by ICD10 code | - | - | 4.350 |
4.762 |
|
| MeanQueryTimeInSeconds | medical cases by ICD9 code | - | - | 11.219 |
12.008 |
|
| MeanQueryTimeInSeconds | medical cases by patient's city | - | - | 10.083 |
11.855 |
|
| MeanQueryTimeInSeconds | medical cases by patient's sex | - | - | 16.016 |
16.509 |
|
| MeanQueryTimeInSeconds | medical cases by patient's age | - | - | 8.205 |
8.688 |
|
| MeanQueryTimeInSeconds | medical cases by visit's dates | - | - | 5.836 |
8.231 |
|
| MeanQueryTimeInSeconds | medical cases by laboratory tests | - | - | 4.316 |
5.227 |
|
| MeanQueryTimeInSeconds | medical cases by all of the above criteria | - | - | 13.467 |
14.019 |
Technical details
A medical case is a list of hospital visits of one patient. A medical case satisfies criteria of a query when at least one visit of a patient satisfies them. MDC provides the following search options and an arbitrary conjunction of them:
- by ICD10 code - a visit satisfies this criteria when a patient has the ICD10 disease as an underlying or a concurrent during the visit
- by ICD9 code - a visit satisfies this criteria when a patient underwent the ICD9 medical procedure during the visit
- by patient’s city - a patient satisfies this criteria when the given city matches to the post code of the patient
- by patient’s sex - a patient satisfies this criteria when the given sex matches to the sex of the patient
- by patient’s age - a patient satisfies this criteria when he or she was in the given age at the time of discharge from the hospital (the age can be specified as an interval)
- by visit's dates - a visit satisfies this criteria when the given period intersects with the visit’s time
- by laboratory tests - a visit satisfies this criteria when all the given laboratory tests were carried out during the visit
- by all of the above criteria - conjunction of all above criteria
Please note that searching by patients‘ attributes is limited to those that remained after anonymization, namely: post code, sex and birth date.
In order to find and compose visits in medical cases two map-reduce jobs are executed. Afterwards, found medical cases are serialized to JSON and returned to the client. The execution time mainly depends on the amount of matching visits and patients. For example ICD9 codes of medical procedures are repeated for many visits. Naturally, the sex is repeated for patients. In the case of more than one criteria, conjunction takes time as well.
The first test (July 24, 2014) was performed when MDC contained information about 16 000 hospital visits.
The second test (July 29, 2014) was performed when MDC contained information about 19 000 hospital visits.
Link to the software that was used to test MDC: https://git.man.poznan.pl/stash/scm/scap/test-scripts/medd-provider-test
Evaluation notes
Could be such things as identified issues, workarounds, data preparation, if not already included above