Universal Data Supply: Know Your Business
As announced in my lessons learned article, I'm starting the series on implementing universal data supply with a concrete business example from the industry.
You may feel like reading about business matters is lengthy and exhausting. Instead you just want to get straight into the detailed steps to migrate your traditional Data Architecture towards the new approach. But let me try to convince you that thoroughly understanding your business is paramount for any architectural changes to truly create business value.
It's not like that you need to fall in love with your business straight away. But I can promise you that if you really get engaged with what you want to develop an IT solution for, your solutions will be much better.
Of all the projects I've worked on over the years – including exciting but sometimes quite abstract solutions for banks and insurance companies – the following was the closest to my heart.
So let's take a look at an illustrative, simplified example derived from a real healthcare project I had the pleasure to work on. It will help us to recognize the potential of universal data supply to create an architecture that solves real business challenges.
It is an example where data technology can actually help save lives. What could be more motivating for us as data engineers?
CleanHands Medical Center
Our fictional hospital ‘CleanHands Medical Center' has a strong commitment to infection prevention and attaches particular importance to thorough hand hygiene.
The business case
Hospital acquired infections (HAI) harm patients every day across healthcare systems worldwide. These infections, many of which are caused by multidrug-resistant organisms, place a significant burden on hospitals and ultimately can cost the lives of patients.
Because of the accelerated rise of anitmicrobial resistance (AMR) the treatment of these infections is hard, causing enormous additional costs for hospitals. These costs include extended hospital stays, increased use of medications, specialized treatments in intensive care units (ICU) and not least of all reputational damage and legal costs. Above all it further increases the risk of disease spread, severe illness and consequently even death of patients.
By investing in effective infection prevention and control (IPC), hospitals can significantly improve patient safety while also reducing costs. According to the WHO "Global Strategy on Infection Prevention and Control", the most effective measure against HAIs is good hand hygiene.
For me personally at least, it was surprising that a procedure as simple as washing your hands or rubbing them with disinfectant is still the most effective way to prevent these extremely harmful infections.
According to current studies, even in modern hospitals in industrialized countries hand hygiene compliance rates are still very low. Average compliance is around 40% across general hospital units. In intensive care units, it tends to be slightly better at around 60%. These figures highlight the ongoing challenges in ensuring consistent hand hygiene – it's a real threat to us all.
Studies have also demonstrated that hand hygiene interventions can achieve a 25–70% reduction in HAI rates. Hand hygiene is not only effective but also significantly reduces costs for hospitals. For every $1 invested it saves approximately $16.5 in healthcare expenses.
Hand hygiene compliance
But why is there still a problem with something as obviously important and simple as maintaining good hand hygiene? After all, there are highly qualified healthcare professionals working in our hospitals who are fully aware of this danger.
Various studies on hand hygiene compliance have shown that it's very often the lack of immediate feedback as an incentive that discourages health-workers from complying. In the daily hustle and bustle and especially in stressful moments, e.g. when there is a shortage of staff due to many sick days, they simply forget it or just skip good hand hygiene due to carelessness.
Repetitive tasks like handwashing, though simple, can also lead to oversight due to fatigue or desensitization over time. Surprisingly, even doctors, who could be assumed to be the most sensitive to this issue, are among the groups with the worst compliance.
Consequently, to promote and maintain proper hand hygiene, hygienists should regularly monitor quality, provide timely feedback to healthcare workers and conduct regular training and assessments.
However, the monitoring process for hand hygiene practices is typically still based on trained observers who directly observe healthcare staff during routine patient care. Yes indeed, there are hygiene specialists on the ward with a clipboard, watching the staff at work.
Observers try to be as discreet as possible to avoid the Hawthorne effect, where staff improve behavior only because they are being observed. But honestly, this effect just can't be avoided completely in this setup.
The observation task itself requires a high level of concentration and accuracy. I tried it myself and failed miserably. For each observed treatment, hygienists must count the number of actual hygiene actions and at the same time keep track of the 5 most important opportunities or moments when such an action would have been necessary.
With this information they can calculate a compliance rate by dividing the number of hand hygiene actions observed by the total number of hand hygiene moments recorded.
As this manual process is error-prone and obviously costly, it can only be carried out on a random basis. The hygienists therefore seek to significantly improve and automate this process to enable ongoing discreet monitoring of hand hygiene compliance.
Smart technology helps to save lives
Especially timely feedback to the staff on their compliance and an enabling environment with operable disinfectant dispensers have been identified as important interventions.
The hospital therefore invested in smart disinfectant dispensers coupled with a near-realtime monitoring system. These smart dispensers can transmit usage data on when and how the device was used together with the amount of liquid dispensed.
The dispenser device establishes the connection to the monitoring system via a standard WLAN protocol that is normally already available in the IT infrastructure of hospitals. The monitoring system receives usage data sent by the devices and stores it in a relational database system. The system offers all kinds of maintenance functions to register dispensers at their specific install location on the wards and to manage them with additional information like the assignment to the organizational units in the hospital. Various reports are available that show current and history dispenser usage metrics on different aggregation levels.
Traditional data architecture for initial improvement
Isolated usage information related to general target values determined in the random observation samples already provides great value for manually deriving a hospital-wide key figure for compliance. However, the hygiene specialists quickly identified further potential for improvement. They wanted to automate the determination of compliance rates on a more granular level.
The hospital IT data engineers have therefore developed a batch-oriented ETL process that extracts all usage data from the dispenser system's relational database into the hospital's Data Warehouse on a weekly basis. This was implemented to combine the dispenser usage data with patient occupancy data and treatment data from the hospital's information system (HIS). The HIS data is also loaded into the data warehouse on a weekly basis via a different batch process.
With the help of some elaborate business rules, the data engineers were able to implement a transformation logic to derive the "expected frequency of use" for all dispensers during a week at ward level. To this end, the hygienists defined a flat-rate number of hygiene actions that are assumed necessary for a specific treatment of a patient. Combined with weekly occupancy it's possible to calculate a continuous weekly compliance rate at the ward level.
They compiled weekly reports on hand hygiene compliance for each individual ward. With this information presented by the respective ward manager, hospital staff were further encouraged to improve their compliance.
This intervention has led to a significant reduction in infections in the hospital. Due to this great success, the hygiene specialists now want to achieve further improvement.
Further improvement
To further incentivize the staff to stay compliant at all times, the hygiene specialists want to significantly reduce the time for feedback to health-workers on their compliance.
Instead of weekly paper reports published to the ward manager, they intend to give more timely feedback to the staff via big monitors. They shall visualize the current state of compliance in the ward with easy to understand traffic light indicators.
On the other hand, it was reported that the nearest dispensers were often just empty and the necessary hand hygiene was skipped due to the extra time it took to leave the room and find the next functioning dispenser.
To solve this problem, technical maintenance staff need to receive near real-time information on the fill level of each dispenser installed. This would improve refill management and thus keep the devices operational at all times.
The long-term vision
To further strengthen the prevention control, the hygienists intend to implement near real-time and individual feedback to the healthcare workers.
Individualization of feedback
Instead of general reports on ward-level the reports and dashboards should be further personalized. Although this example is fictional, it's important to understand necessary security aspects you need to pay attention to when working with tracking data or potential private personal information (PPI). Any information that could be used to establish a link to an individual's identity or violate other privacy concerns must be protected from unauthorized access.
- Compliance reports and dashboards showing their hygiene performance, improvement opportunities and individual compliance per healthcare worker compared to hospital benchmarks.
- Gamified tracking where healthcare workers earn points for correct hand hygiene. This could include rewards or milestones to motivate ongoing compliance.
Smart and discreet technology to reduce feedback time
The idea is here to have each healthcare worker equipped with smart wearables (like watches) that are able to receive real-time, individual and discreet feedback from the monitoring system.
The intention is to incentivize correct hand hygiene right at the point of care whenever the healthcare worker starts a patient treatment. This would include visual and/or haptical feedback from the smart wearable as soon as the system detects one of the 5 hand hygiene moments.
Here are some of the advanced ideas using smart sensors and future technologies for further improving hand hygiene compliance.
- Advanced sensors in dispensers to detect non-compliant hand movements (e.g., not using enough liquid) and send instant feedback to the smart wearables.
- Use patient-specific hand hygiene recommendations based on treatment information from the HIS, when entering the patient room.
- Use ML models based on historical compliance data to predict potential non-compliant events. These models could send pre-emptive reminders to staff at high-risk moments based on real-time data from different monitoring systems.
- Create sanitization zones around hospital beds where hand hygiene compliance is tracked using spatial sensors and granular treatment information of the individual patient. Smart wearables could interact with these zones to prompt compliance as soon as a healthcare worker enters the area, all based on real-time occupancy and hygiene data.
The next step after this initial business case analysis is an assessment of the current architecture in terms of its ability to support the short and medium-term goals and its potential to realize the long-term vision.
We haven't discussed every detail yet, but we have a good understanding of the overall business case. With this overview, we can start to create an initial plan to gradually adapt the current architecture. When we further discuss our solution proposal with the hygienists, we can work out the steps together with more details we'll learn about the business case. Stay tuned for more insights!
