Decision-makers and healthcare managers must introduce novel healthcare technologies without losing sight of the economic sustainability of the system.1 Different tools and methodologies for planning, prioritization, and allocation of resources are available for decision-making, including budget impact and economic evaluation.2 However, these tools do not consider criteria such as equity or burden of disease, or at least not explicitly.3–5

In the field of health technology assessment, multicriteria decision analysis (MCDA) is emerging as a promising tool.3 MCDA is a set of methods that, complementing other techniques, aids decision-making in evidence-based prioritization and use of resources and enables practitioners to address complex problems across multiple dimensions.5 The MCDA is a methodology that explicitly orders and assists in deliberative processes, i.e., it provides support without prescriptive value. Thus, it incorporates different values and interests in an explicit and transparent manner, leading to improved engagement, transparency, and accountability.6–8 The growing literature on MCDA in healthcare decisions points to three reasons why this group of methods may represent an alternative to economic evaluation in STD procedures.9,10 First, the explicit inclusion of a list of value dimensions; second, the assignment of quantitative weights among the different evaluation criteria, so that their relative importance is also explicit, which improves the transparency of the preference bidding process and the participation of the agents involved; and third, the possibility of including all of them in the value assessment, which contributes to the legitimacy of the process.9,10

The aim of this study was to evaluate the value of outsourced versus in-hospital dialysis service provision for the treatment of chronic kidney disease (CKD) in Spain using the MCDA methodology. The advantage of comparing these two types of service provision with MCDA is that it allows us to incorporate multiple elements, beyond efficacy, safety, and price in making decisions on financing, provision, and prioritization of health resources, when presenting two alternatives in the provision of a care service.

In a multi-criteria decision context, the main issues are: 1) problem structuring and identification of alternatives; 2) evidence review; 3) modeling (identifying and establishing evaluation criteria for the different alternatives; measuring and prioritizing interventions according to the established criteria; weighting to obtain the relative importance of each criterion and an overall evaluation of each intervention) and 4) deliberation.3

The development of the MCDA workshop was carried out virtually and used the EVIDEM (Evidence and Value: Impact on DEcision-Making) framework, which is freely available through its website (The EVIDEM Collaboration).11 This framework consists of 22 criteria (15 quantitative and seven qualitative) and requires that evidence meet three characteristics: 1) broad (covering all areas related to the health issue in question); 2) relevant (key to the decision) and 3) explicit, as a key element of transparency and accountability in decision-making.

To carry out the MCDA, a group of 23 participants was formed, balancing different levels of responsibility, from the Andalusian Public Health System (SSPA) made up of directors and managers (n = 3), representatives of the Association for the Fight Against Kidney Diseases (ALCER) (n = 2), health evaluation and management professionals (n = 5), healthcare practitioners (n = 3), nephrology nursing professionals (n = 5), and nephrology unit managers (n = 3). In addition, to include other organizational perspectives, two supervising nephrologists from other autonomous communities were invited.

In the first part of the workshop, participants were offered a training session on the MCDA methodology as a decision-making tool by an expert in health technology assessment.

In the second part, and before the participants were familiarized with the methodology under study, weights were assigned to the evaluation framework: each participant scored each criterion according to importance between 1 (least relevant) and 5 (most relevant).

Subsequently, participants were presented with the results of the scoping review of the literature on the subject, according to the different domains of the evaluation framework.12 For this review, the following databases were consulted: PubMed, Embase, Web of Science, CINAHL, Scopus, and The Cochrane Library. We included articles comparing outsourced versus hospital dialysis, according to the EVIDEM framework criteria.

After this, each participant scored the absolute criteria (those not involving a comparison between alternatives) between 0 (lowest value) and 5 (highest value) and the relative criteria (involving a comparison between both dialysis delivery modalities) between −5 and 5 points. Finally, an evaluation was made of the 7 qualitative criteria, based on the available evidence and the experience of the participants.

The estimated value for each intervention was obtained using a linear additive model, combining the relative weighting of each criterion with the score reported for each intervention. Each estimated value was transformed into a scale from 0 to 100, to facilitate its interpretation and its standard deviation (SD), which represents the synthesis of the overall value of each alternative as an expression of the degree to which one of them is preferred to another. To check the degree of consistency and replicability of the analysis, the model was weighted by scoring the criteria from 1 to 5.

Finally, although different programs are available for multicriteria decision analysis,13 we designed a tool in Microsoft Excel format that allowed the information collected to be obtained and displayed in a more explicit, dynamic, and transparent way. At the end of the workshop, the participants discussed the results, and held a final debate on the contribution of this methodology and its application to decision-making in different areas of nephrology.

In the first part of the model weighting, workshop participants scored all criteria above 3 (out of 5), except the indirect cost criterion, which scored 2.8 (SD: ±0.8). The highest mean score was 4.3 for the criteria “comparative effectiveness” (SD: ±0.8) and “comparative safety/tolerability” (SD: ±1.0) (Fig. 1). We observed high consistency between the scores using the 0–5 scale and the 0–100 scale (Figs. S1A–S1F in Appendix A show the mean score according to professional profile).

Fig. 1.

Average weight of the EVIDEM value assessment on a scale of 1–5.

(0.29MB).

With regard to the absolute criteria scores, in the hospital dialysis option, the criterion with the highest mean score was “unmet needs” with 3.8 (out of 5) and high consistency (SD: ±0.8); the other criteria with the highest mean scores were “severity of disease,” “quality of evidence,” and “type of therapeutic benefit” with a score of 3.3, with a standard deviation ranging from 0.8 to 1.2. In contrast, the criterion with the lowest score was “type of preventive benefit” with score 2.3 (SD: ±1.0). As for the comparative criteria, the criterion with the highest score was “comparative safety/tolerability” with a mean score of 1.2 (SD: ±2.6); the remaining criteria ranged from 0.3 (SD: ±3) for “patient-reported outcomes” to −0.2 (SD: ±3.1) for “indirect costs” and “fixed health care costs” (SD: ±2.5).

In the outsourced dialysis option, the highest mean score was 3.7 (SD: ±1.2) for the criterion “size of the affected population,” followed by “unmet need” with a score of 3.5 (SD: ±0.5). In contrast, the lowest mean score was for "type of preventive benefit” with a mean of 2.3 (SD: ±1.5). The relative criteria scores ranged from 0.8 (SD: ±3.3) for the criterion “comparative costs of the intervention.” The lowest were for “compared efficacy/effectiveness” and “compared non-health costs,” with a score of −0.8.

When the scores were analyzed according to the profile of the participants, it was observed that, in general, executives and managers gave a higher score to those dimensions related to cost, whereas patients gave them the lowest score. In contrast, the “quality of evidence” item scored lowest among patients and had the highest mean score among evaluators (Figs. S1A–S1E in Appendix A).

The overall value estimate integrates the weights and scores of each panelist on a scale from 0 to 100. Thus, Figs. 2 and 3 show the mean contribution of each criterion to the overall value estimate for hospital and outsourced dialysis, respectively. The estimate of the value of the outsourced dialysis was 0.29 (SD: ±0.2) and for hospital dialysis 0.39 (SD: ±0.2). When analyzing the mean score of the overall value, according to professional profile, it can be seen that all the profiles gave a higher value to hospital hemodialysis (range: 0.44–0.36 for patients and managers, respectively) than to conventional dialysis (range: 0.31–0.27 for patients and middle management, respectively) (Figs. S2A–S2E in Appendix B for hospital dialysis and Figs. S3A–S3E for conventional dialysis, according to professional profile).

Fig. 2.

Contributions of mean score of each quantitative criterion and estimates of the overall score of hospital dialysis.

(0.58MB).

Fig. 3.

Contributions of mean score of each quantitative criterion and estimates of the overall score of outsourced dialysis.

(0.58MB).

Finally, the workshop participants evaluated the impact of each qualitative or contextual criterion,11 with the criterion “common objective and specific interests” showing the greatest negative impact (12 participants rated it as such). On the other hand, “opportunity cost and affordability,” and “population priorities and access were the two criteria that showed the greatest positive impact” (Fig. 4).

Fig. 4.

Impact of the qualitative scores assigned by the participants in the evaluation.

(0.16MB).

MCDA is increasingly used worldwide for health care decisions to improve the consistency and transparency of policy decisions.14 MCDA applications can be classified into two categories12,15: 1) resource allocation, assigning priority based on the outcome of MCDA application16 (reimbursement and authorization decisions or budget allocation) and 2) ranking therapeutic alternatives for specific clinical problems, developing clinical guidelines or incorporating patient preferences in shared decision-making.

This article presents an assessment, using the MCDA methodology, of two non-exclusive alternatives in the field of the provision of treatment for a highly prevalent chronic disease whose assessment does not imply the allocation of resources toward one modality or the other. To the authors’ knowledge, there is little experience in the application of this methodology for decisions of this nature applied to the field of nephrology.

The complexity of the healthcare sectors, budgetary constraints, and the need to offer high quality services have led healthcare organizations to adopt new management approaches. Thus, outsourcing of services has been considered in recent years by health care decision-makers as one of the tools for improving organization, development, and productivity. The outsourcing of services is a mechanism for assigning some of the organization’s activities to an external provider. Among the various reasons for outsourcing are: cost reduction, increased efficiency by focusing on value-adding processes, improved skills, reduced service delivery time, and increased competitive advantage.17,18

The outsourcing of services is especially relevant in the treatment of chronic kidney disease; in Spain, public–private collaboration plays an important role in the dialysis provision model and many patients receive hemodialysis in out-of-hospital clinics, through agreements with the different health services.12

The prevalence of chronic kidney disease in Spain is around 15% and the results have shown that it can be considered a cardiovascular condition.19 Moreover, CKD represents a very high cost20: it accounts for 3% of total health expenditure and requires health care that involves a multitude of health professionals at different levels, with a high level of coordination for adequate prevention, diagnosis and treatment.21 Bearing in mind that we are situated in a setting characterized by the high cost to the health system of treating chronic renal failure, this work has approximated a measure that compares the value that two forms of service provision of dialysis through out-of-hospital clinics and hospitals can provide. A similar initiative developed a multicriteria evaluation methodology for dialysis centers based on the value of health care and understood as the ratio of patient benefit per monetary unit invested. Four criteria were used (evidence-based clinical performance, annual mortality, patient satisfaction, and health-related quality of life) and MCDA was used in the phase of assigning weights or weighting of criteria and sub-criteria. Although the number of external dialysis clinics included in the study was not high (n = 5), the three clinics that provided the highest value were all external.22

Regardless of the results, there is no doubt that a value approach, through MCDA or other similar methodology, has the potential to integrate different perceptions of all stakeholders and create a context for improving a treatment or the way in which a care service is provided, innovate processes, pay for performance, and improve transparency. Moreover, the results can be easily understood by the different stakeholders involved (by patients and their families, dialysis center health professionals, managers, providers, or political decision-makers) and are reproducible in different evaluation contexts.

At the end of the workshop, there was a space to collect the opinions of the participants on the applicability and utility of this methodology. In general, the panelists affirmed that it is a useful and interesting exercise and that, to a certain extent, it provides security in decision-making, since it allows different criteria considered to be ordered, justified, and explicitly considered. They also stated that the exchange of opinions among the participants enriched the process, especially because of the diversity of profiles.

These aspects are particularly relevant in the context of healthcare organizations, which are characterized by a high level of complexity and a very dynamic environment.23 Although the participants were offered a theoretical session on MCDA and an attempt was made to adapt the language to all the panelists, they reported difficulties with the terminology used, which required a certain familiarity, and uncertainty interpreting the evidence and the type of concepts used,24 especially among those with less experience in health technology assessment.

This work has several limitations that may affect the results of the study. First, due to the pandemic, the workshop was conducted virtually, which may have affected the interaction between participants and the discussion generated. On the other hand, the uncertainty and care overload generated during the pandemic made it impossible to carry out a subsequent retest to assess the consistency of the scores.25

The EVIDEM evaluation framework was used for this evaluation, but it is possible that it omitted certain criteria that might have been relevant such a specific evaluation as the one proposed in this work.26 Nevertheless, three professionals not participating in the workshop were asked for a prior individual evaluation in order to consider the adequacy of the criteria considered in the workshop.26 The EVIDEM framework was considered to cover all aspects relevant to the evaluation, did not require complicated calculations, and there was experience in its use.27

The relevant aspect of the result is not the absolute values obtained, but the comparison between the two values. Thus, the main contribution of the MCDA is the deliberative process itself and the discussion of the results.3,28 Thus, according to this study, the fact that professionals give more value to hospital dialysis than to conventional dialysis should translate into a prioritization in the allocation of health resources, decision-making and formulation of health policies.

The Andalusian School of Public Health (Escuela Andaluza de Salud Pública), the institution responsible for the management and development of the project, received funding from Vifor Pharma. Vifor Pharma had no role in the study design, data collection and analysis, decision to publish, or preparation of the manuscript.

The laboratory that funded this work did not participate or intervene in the group’s decisions including the choice of the problem, selection of workshop participants, or in the analysis, writing, presentation, and approval of the manuscript. The authors of this article declare that they are not subject to any conflict of interest related to the subject that may affect the design, analysis, or presentation of results.