Journal Information
Vol. 32. Issue. 5.September 2012
Pages 555-701
Vol. 32. Issue. 5.September 2012
Pages 555-701
Full text access
Outcomes weighting for comprehensive haemodialysis centre assessment
Ponderación de resultados para la evaluación global de centros de hemodiálisis
Visits
6971
Gestión de la Calidad de la Sociedad Española de Nefrología, Eduardo Parrab, M. Dolores Arenasc, Manuel Alonsod, M. Fernanda Martíneze, Ángel Gamenf, Sagrario Baldag, Pablo Rebolloh, Tomás Martínezi, Fernando Álvarez-Udej
b Servicio de Nefrología, Hospital Reina Sofía de Tudela, Tudela, Navarra, España,
c Servicio de Nefrología, Hospital Perpetuo Socorro, Alicante, España,
d Servicio de Nefrología, Hospital Valle del Nalón, Langreo, Asturias, España,
e Servicio de Nefrología, Hospital Casa de la Salud, Valencia, España,
f Servicio de Nefrología, Hospital Reina Sofía de Tudela, Tudela Navarra, España,
g Servicio de Nefrología, King's College Hospital, Londres, Reino Unido,
h Servicio de Nefrología, BAP Health Outcomes Research, Gijón, Asturias, España,
i Departamento de Bioestadística, Universidad de Zaragoza, Zaragoza, España,
j Servicio de Nefrología, Hospital General de Segovia, Segovia, España,
Ver más
This item has received
Article information
Abstract
Full Text
Bibliography
Download PDF
Statistics
Figures (2)
Tables (1)
expressing the homogeneity and contingency among different intra-actor weightings; the third also includes the results for all three actors together

Antecedentes: En la actualidad, no disponemos de un sistema de evaluación de centros de diálisis adecuado. Objetivos: Construir una estructura de ponderación global de resultados de hemodiálisis, aceptable por los diferentes agentes y que permita la comparación de centros mediante un indicador compuesto. Métodos: El Grupo de Trabajo de Gestión de la Calidad de la Sociedad Española de Nefrología (GT) estableció un conjunto de indicadores preseleccionados. Se constituyó un Grupo Focal (GF), independiente del GT, compuesto por nueve individuos; tres pacientes, tres clínicos y tres gestores sanitarios, que con una metodología reglada valoró dichos indicadores y estableció los indicadores seleccionados. Posteriormente, realizó una ponderación de los indicadores mediante tres rondas de ponderación, separadas cada una de ellas por dos períodos de debate, consistentes en la distribución de 100 puntos entre cada variable, de acuerdo con sus valores personales y el debate sostenido. Resultados: Los resultados clínicos incluyeron: dosis de hemodiálisis, anemia, calcio y fósforo plasmático, tipo de acceso vascular y días de hospitalización. El peso otorgado a cada variable tras la tercera ponderación, expresado como la media de todos los agentes, fue: resultados clínicos, 38,9; mortalidad anual, 25,0; satisfacción con el centro, 12,2; calidad de vida relacionada con salud, 15,6; y costes, 8,3 (total 100). Conclusiones: La estructura de ponderación abarca resultados relevantes y globales e incluye la perspectiva de los agentes involucrados; todo ello puede incrementar su aceptabilidad y difusión, así como contribuir al análisis del valor producido por los centros y a la mejorara de los resultados.

Palabras clave:
Satisfacción de pacientes
Palabras clave:
Calidad de vida
Palabras clave:
Calidad asistencial
Palabras clave:
Evaluación de resultados
Palabras clave:
Hemodiálisis
Palabras clave:
Insuficiencia renal crónica terminal

Background: At present, there is no adequate system available for evaluating dialysis centres. Objectives: To construct an overall haemodialysis results weighting system, acceptable to the different stakeholders involved which allows the comparison of centres using a compound marker. Method: The Quality Management Work Group of the Spanish Society of Nephrology (WG) established a set of preselected indicators. A Focus Group, independent of the WG, was established. It was made up of nine individuals: three patients, three clinicians and three clinical managers, who assessed these indicators using an approved methodology and established the selected indicators. Finally, the indicators were weighted through three weighting stages, each separated by two debate periods, which involved the distribution of 100 points between each variable, according to the personal assessment and the debate sustained. Results: The clinical results included: haemodialysis doses, anaemia, plasma calcium and phosphorus, type of vascular access, and hospitalization days. The weighting given to each variable following the third weighting process, expressed as an average of all the factors, was as follows: clinical results 38.9; annual mortality 25.0; satisfaction with the centre 12.2; health-related quality of life 15.6; and cost 8.3 (total 100). Conclusions: The weighting structure covers relevant and overall results and includes the opinion of all stakeholders involved; all of which may increase its acceptability and widespread use and contribute to the analysis of the value produced by the centres and the improvement of the results.  

Keywords:
Patient satisfaction
Keywords:
quality of life
Keywords:
quality of care
Keywords:
results evaluation
Keywords:
Haemodialysis
Keywords:
end-stage chronic renal failure
Full Text

INTRODUCTION

 

There are currently more than 180 000 patients on haemodialysis (HD) in Europe, receiving treatment at 4000 centres.1,2 Researchers have observed a significant variability in the results from treatment between different centres, related to heterogeneous quality of the care provided to patients.3-6 In addition, HD costs in Spain range between 30 000 and 47 000 Euros per patient per year.7,8 However, despite the great social, health, and economic importance of renal replacement therapy, we currently do not have access to an adequate system for evaluating dialysis centres.

The Quality control Working Group of the Spanish Society of Nephrology (WG) developed this study for the purpose of evaluating HD centres reflecting the values and preferences of stakeholders. The objective is to create a system for weighting the results from each HD centre from a global health perspective, taking into account: clinical results, mortality, health-related quality of life, patient satisfaction, and costs, and thus facilitating the creation of an adequate compound indicator for comparing results between centres acceptable among dialysis stakeholders.

 

METHOD

 

This study forms part of the Global Haemodialysis Centre Evaluation Study developed by the WG, whose final objective is the effective evaluation of these centres by analysing the corresponding values produced.

WG first established essential requirements for an “appropriate” HD centre evaluation system; plain, transparent, comprehensive, reproducible, acceptable, benchmarking, useful, and focused on centre improvement.

The WG, composed exclusively of clinicians and based on previous studies,9,10 established a first group of indicators for the results of HD treatment referred to as preselected indicators. Later, a Focus Group (FG) was established, independent of the WG, which was composed of nine individuals who represented the primary actors in the HD process; three patients, three clinicians, and three clinical managers.

The requirements for the different constituents of the FG were: patients must have been on HD for at least three years and must have served as coordinators for renal patient organisations; the clinicians must have been of recognised prestige and ample experience, two of these nephrologists and one an internal medicine specialist; the clinical managers were selected for three different specialties: economic, medical, and health services researcher.

In order for the FG to carry out its work, we established a standardised methodology to be carried out in different phases.11,12 Once the members of the FG accepted participation, and before any previous contact had been made between them, each received an email with a document outlining the purpose of the study and the preselected indicators. Two weeks after receiving the document, all members of the FG met to receive more information regarding the study framework, the objectives of the study, and the preselected indicators. This was followed by a discussion to evaluate the incorporation, elimination, or modification of several of the preselected indicators in order to obtain selected indicators. This in turn was followed by three rounds of individual weighting of the selected indicators, each of which was separated by a session of debate and reflection. As such, two periods of discussion were held, one between the first and second round of individual weighting, and the second between the second and third rounds. The individual weighting process consisted of allotting each member of the FG with 100 points to assign to each of the selected indicators based on their personal values and preferences, taking into account the insights gained in the discussion periods. The debate carried out during the discussion periods was based on contrasting the different weighting results that varied between different individuals and actors. The second round of individual weighting could result in the approval of a metric in the case of unanimity among all members. If the decisions were not unanimous after the second round, a third round of individual weighting continued, whose mean result was determinately approved. The meetings were directed by a member of the WG, recorded, and evaluated a posteriori by two independent clinicians.

We established the requirements that the clinical results should comply with: routine use in HD centres, therapeutic resources capable of modification, and a direct relationship with morbidity/mortality.13-15 The clinical results were weighted by group evaluation, although individual comments on each result from each actor were encouraged and documented. Later, in the WG, we considered the importance of each different indicator that was unanimously approved.

 

RESULTS

 

The preselected indicators were established by the WG in November 2007, and are summarised in Table 1. During December 2010, once the members of the FG received information regarding the study characteristics and preselected indicators, the group was formed in agreement with the established requirements. During the FG meeting, in the evaluation phase of the preselected indicators, a new indicator was proposed in the form of the number of days of hospitalization per patient per year. This indicator was suggested by the clinicians, and was unanimously approved for inclusion. The FG carried out the first and second rounds of individual weighting of the selected indicators, and when no unanimous decisions were made in the second round, a third was required as well. The lack of unanimity after the second round of debate was due to several members claiming that mortality was not given sufficient weight. The weightings of the different variables in all three rounds are shown in Table 2. To simplify, the results of the weighting are presented as the mean value for each facet of the focus group: patients, clinicians, and managers.

During the second discussion period, and by initiative of the patients group, the importance of certain clinical indicators was argued over others. From their perspective, the type of vascular access was a particularly relevant variable due to the greater level of incommodity and decreased quality of life associated with using a central venous catheter as compared to an arteriovenous fistula. Patients requested that this consideration be taken into account when weighting clinical results. For their part, clinicians seconded the proposal of considering the type of vascular access as a critical point in obtaining adequate results on HD. The meeting lasted a total of 210 minutes, separated into two periods on the same day.

With successive rounds of weighting, clinical results and annual mortality rates tended to gain more importance as compared to initial values, whereas other results had an inverse tendency. The third weighting session received the unanimous approval of all members of the FG. The weighted variables, presented as a mean of all components of the FG were: clinical results: 38.9; annual mortality: 25.0; patient satisfaction with the HD centre: 12.2; health-related quality of life: 15.6; and economic cost: 8.3 (total: 100).

 

DISCUSSION

 

Evidence-based medicine and the evaluation of health technologies are primarily focused on improving the benefits to the patient. However, the indicators used tend to be chosen by researchers who do not take into account the opinions of patients or other actors in the provision of health care.16 This frequently leads to indifference or even rejection of evaluation systems by patients, clinicians, and managers. An incorporation of the opinions of different actors, especially patients, and integration of their points of view should be a fundamental aspect of evaluating the results in any healthcare process. Democratisation of evaluation systems is an essential element for facilitating their acceptance and achieving satisfactory results.

The models used for certifying and accrediting dialysis centres assume compliance with certain requirements and standards without considering the opinions of the affected actors in the healthcare process. Additionally, these models are inadequate for establishing comparisons between different centres. On the other hand, the model proposed by the European Foundation for Quality Management (EFQM) assigns a disproportionate value to process indicators (called agents), to the detriment of variables reflecting clinical results. The process indicators should be considered as one tactic for the achievement of a primary objective, which comes in the form of the final results, and as being proportionate to the value produced.17

We believe that the creation of a compound indicator can facilitate comparisons between different centres (benchmarking) and aid in the effective evaluation of the value produced by each of them. An indicator of these characteristics can maximise the stimulus for improvement in these centres. Such a model can also allow for desegregating the different components of the indicator, thus avoiding a loss of information which accompanies the use of a single indicator. When the information is obtained on an individual basis, we can also adjust for comorbidity between centres. As such, the model would combine the stimulus produced by a single indicator with the availability of disaggregated information, focused on centre improvement. Such a model is comprehensive because it considers all variables relevant to the process of HD. Associated with a previously established automatic classification system for the results at each centre, the model is also simple and capable of providing transparency and reproducibility in the evaluation of HD centres.

The methodology used by the FG was chosen because it provides a useful tool for obtaining and processing information based on complex and qualitative sources, with ample detail and diverse backgrounds. Our methodology necessitated a capacity for integrating different points of view, beliefs, and experiences, which is a distinct possibility when using a FG. In order to achieve the objective of weighting the different variables, the primary focus of the study, we did not consider it necessary to perform a quantitative analysis of the results; however, we did use a standardised methodology.

As regards the evolution of the individual weighting process, clinical results and annual mortality progressively gained more importance in the second and third weighting sessions as compared to the first, whereas patient satisfaction and health-related quality of life lost importance. This progression could suggest a tendency to assign a greater value to objective sources of evaluation as the comprehension of these variables increases. One notable aspect that upholds this interpretation is the fact that the initial weightings, which came before the first discussion and debate, were not radically different between the different actors, but rather quite comparable, and with a maximum difference between groups of 10 points out of 100. In addition, the results of the second and third weighting sessions were similar between the three groups of actors, which demonstrates the level of consensus reached through discussion and agreement of opinion following debate. This progression suggests that the points of view held by different actors are not so disparate once all of their members deeply understand the significance of the results and can reflect upon them.

However, the study also had various limitations. Firstly, the representativeness of each group may be insufficient for establishing the actors’ opinions. However, the differences between the final intra-actor weights (patients, clinicians, and managers) were minimal (the results are not shown for simplification). The difference reached 12 points in one variable, 10 in three, and less than 10 in the rest, which indicates substantial internal validity. On the other hand, since all members of the FG came from a single cultural context, we cannot extrapolate the results to other areas with different sets of values.

To conclude, the weighting structure of the HD results appears appropriate for evaluating HD centres, since it takes into account the relevant results from a comprehensive perspective and assigns weight in accordance with the values and preferences of the actors involved in the process. An evaluation of these characteristics may result more acceptable for patients, clinicians, and managers, as well as providing greater legitimacy for effective implementation and dissemination, while also contributing to the evaluation of the value produced in each centre and improving their results.

 

Conflicts of interest

The authors state that they have no potential conflicts of interest related to the contents of this article.

Table 1. Indicators preselected by the Quality Management Working Group

Table 2. Weighting of the selected indicators, performed by patients, clinicians, and clinical managers in the first, second, and third rounds

Table expressing the homogeneity and contingency among different intra-actor weightings; the third also includes the results for all three actors together

Bibliography
[1]
European Commission, Eurostat, Total population. Available at: http://epp.eurostat.ec.europa.eu/tgm/table.do?tab=table&language=en&pcode=tps00001&tableSelection=1&footnotes=yes&labeling=labels&plugin=1 [Accessed: July 5, 2012].
[2]
Kramer A, Stel V, Zoccali C, Heaf J, Ansell D, Grönhagen-Riska C, et al; ERA-EDTA Registry. An update on renal replacement therapy in Europe: ERA-EDTA Registry data from 1997 to 2006. Nephrol Dial Transplant 2009;24:3557-66. [Pubmed]
[3]
Reddan DN, Frankenfield DL, Klassen PS, Coladonato JA, Szczech L, Johnson CA, et al. Regional variability in anaemia management and haemoglobin in the US. Nephrol Dial Transplant 2003;18:147-52. [Pubmed]
[4]
Fink JC, Zhan M, Blahut SA, Soucie M, McClellan WM. Measuring the efficacy of a quality improvement program in dialysis adequacy with changes in center effects. J Am Soc Nephrol 2002;13:2338-44. [Pubmed]
[5]
Saudan P, Halabi G, Perneger T, Wasserfallen JB, Kossovsky M, Feldman H, et al. Variability in quality of care among dialysis units in western Switzerland. Nephrol Dial Transplant 2005;20:1854-63. [Pubmed]
[6]
Hecking E, Bragg-Gresham JL, Rayner HC, Pisoni RL, Andreucci VE, Combe C, et al. Haemodialysis prescription, adherence and nutritional indicators in five European countries: results from the Dialysis Outcomes and Practice Patterns Study (DOPPS). Nephrol Dial Transplant 2004;19:100-7. [Pubmed]
[7]
Parra Moncasi E, Arenas Jiménez MD, Alonso M, Martínez MF, Gamen Pardo A, Rebollo P, et al.; Grupo de Gestión de la Calidad de la Sociedad Española de Nefrología. Multicentre study of haemodialysis costs. Nefrologia 2011;31(3):299-307.
[8]
Villa G, Rodríguez-Carmona A, Fernández-Ortiz L, Cuervo J, Rebollo P, Otero A, et al. Cost analysis of the Spanish renal replacement therapy programme. Nephrol Dial Transplant 2011;26(11):3709-14.
[9]
Alcázar JM, Arenas MD, Alvarez-Ude F, Virto R, Rubio E, Maduell F, et al. Preliminary results of the Spanish Society of Nephrology multicenter study of quality performance measures: hemodialysis outcomes can be improved. Nefrologia 2008;28(6):597-606. [Pubmed]
[10]
Parra E, Ramos R, Betriu A, Paniagua J, Belart M, Martínez T. Effect of a quality improvement strategy on several haemodialysis outcomes. Nephrol Dial Transplant 2008;23:2943-7. [Pubmed]
[11]
Kitzinger J. Qualitative Research. Introducing focus groups. BMJ 1995;311:299-302.
[12]
Powell RA, Single HM. Focus groups. Int J Qual Health Care 1996;8:499-504. [Pubmed]
[13]
Mainz J. Defining and classifying clinical indicators for quality improvement. Int J Qual Health Care 2003;15:523-30. [Pubmed]
[14]
Rocco MV, Frankenfield DL, Hopson SD, McCellan WM. Relationship between clinical performance measures and outcomes among patients receiving long-term hemodialysis. Ann Intern Med 2006;145:512-9. [Pubmed]
[15]
Plantinga LC, Fink NE, Jaar BG, Sadler JH, Levin NW, Coresh J, et al. Attainment of clinical performance targets and improvement in clinical outcomes and resource use in hemodialysis care: a prospective cohort study. BMC Health Serv Res 2007;7:5.
[16]
Tomes N. Patient empowerment and the dilemmas of late-modern medicalisation. Lancet 2007;369:698-700. [Pubmed]
[17]
Porter ME. What is value in health care? N Engl J Med 2010;363(26):2477-81. [Pubmed]
Idiomas
Nefrología (English Edition)
Article options
Tools
es en

¿Es usted profesional sanitario apto para prescribir o dispensar medicamentos?

Are you a health professional able to prescribe or dispense drugs?