With the Center of Excellence Award Designation on the horizon I thought I would resurrect an abstract from the Annual SPS Conference in 2012 that described how our Pediatric Sedation Service built a Sedation Dashboard and follow with some comments about how the dashboard has changed and lessons we have learned since.
Development of Quality and Safety Dashboard and Metrics for Pediatric Sedation
Carrie E, Makin, RN, BSN; Sarah E Heck, MBA; Cheri D Landers, MD
Background: Metrics evaluating quality and safety indicators for pediatric sedation have been adapted from anesthesia providers; however, these indicators are not readily adaptable to pediatric sedation practiced outside of the operating room by non-anesthesiologists. Although multiple professional groups publish guidelines for the safe performance of pediatric sedation; benchmarks for sedation outcomes are not known. Pediatric sedation has similarities to both the operating room and the outpatient clinic setting making it a unique service line for which benchmarks would be helpful.
Methods: At Kentucky Children’s Hospital (KCH) the pediatric sedation team developed internal benchmarks by utilizing the Pediatric Sedation Research Consortium database and a local collection tool to evaluate performance in the quality and safety arena. A stoplight format dashboard was developed and areas were identified as priorities for improvement. Internal benchmarks in all areas were developed initially from surrogate metrics in other service lines (first case start times similar to the OR and no-show rates as in the outpatient setting). Initial benchmarks have been adjusted over time.
Results: The KCH sedation dashboard contains 5 traditional quality and safety domains (Quality, Safety, Efficiency, Service/Satisfaction and Volume) with multiple metrics under each domain. The sedation team consistently performed within the established goals in the Quality and Safety domains as well as the Patient Satisfaction domain but the Efficiency domain metrics and the IV access metric were identified as areas for opportunity. From this information, a more detailed “root cause analysis” type database was developed in REDCap1 to evaluate the etiology of start time delays. In addition, pain associated with IV access has led to skills evaluation and a topical anesthetic policy review/educational effort.
Conclusions: Internal benchmarks are helpful for service line improvement projects in pediatric sedation. This data from one institution should stimulate the development of national quality and safety benchmarks for pediatric sedation providers worldwide.
1Paul A. Harris, Robert Taylor, Robert Thielke, et al. Research electronic data capture (REDCap) – A metadata-driven methodology and workflow process for providing translational research informatics support, J Biomed Inform. 2009 Apr;42(2):377-81.
Selecting the metrics: The original dashboard that KCH started had 19 metrics; the current version has 35. Although perhaps one or two items have been removed, obviously we have added metrics over the past 5 years. Honestly, there are some more metrics that should probably be removed. For example, monitoring the frequency that reversal agents are needed is not fruitful for a service that uses over 90% propofol alone. Also, we have multiple ways of looking at how well we utilize the time we have available to sedate patients, focusing on how well we schedule into the time available is probably the most important then we can track the no show and cancellation rate to capture other types of utilization like the actual time spend performing sedation. The bottom line is that sometimes too much data can become distracting. Collecting and presenting data just for its own sake doesn’t necessarily lead to improvement.
Selecting targets: In the past many years I believe the pediatric sedation community has come to accept the PRSC database and the publications arising out of it as a source for benchmarks for the outcomes that are collected there. Other quality metrics in the domains of efficiency, timeliness, etc. remain elusive with regard to benchmarks. What this abstract describes as benchmarks are more in line with targets with regard to these other domains. In other words, we may not know what the gold standard/benchmark is for an on time start for pediatric sedation first case MRIs when sedation’s first case isn’t the first MRI of the day, but the team can select a target for the service based on what the current performance is and what it would like it to be.
Making Improvements: The abstract described three areas where the team sought to improve their performance: IV success rates, topical anesthetic use and on time starts. Thankfully, the success reported with IV success rates and topical anesthetic use has continued and even further improved. We continue to be beleaguered with on time start rates that are below target. As mentioned, a root cause analysis was performed and revealed a multitude of causes for delays in case starts. A couple years after this abstract, we embarked upon an 8-Step process to dive even further into possibilities to improve this process. We even went so far as to change our target to a more reasonable goal. Outside environment changes continue to hamper our efforts but it has stayed on the dashboard despite it staying “red” month after month. It is my opinion that just because we haven’t been able to make it to our target in this area we shouldn’t turn our backs on the information.
Challenges: One of the biggest challenges has been keeping the dashboard up to date and timely. The initial data tool was developed by a member of our finance department. It is an Excel file that has calculations built into it to report the percentages and times that then build the dashboard. With the expansion of the sedation service and the expansion of the dashboard the Excel tool has been expanded easily based on the original design but the amount of data being entered has increased. This meant that data entry had to move from the medical director to someone with more dedicated time. The timing for this worked as we were able to add this role to the duties of the PICU research coordinator who was also entering VPS and NEARS4KIDS data. Even with someone dedicated to data entry, there is still oversight needed by the sedation leadership to make sure that the dashboard doesn’t become just a pretty rainbow of colors and that it stays meaningful and helps to direct quality improvement efforts.