External Validation of LUMPO3

Approximately 50% of patients with uveal melanoma develop metastatic disease, usually
occurring in the liver (1). Patients’ survival is directly related to the presence of hepatic metastasis.
After the onset of symptoms, most patients with metastatic disease die within a year, and the different
forms of existing treatment seem to rarely extend life considerably (2).

There are some suggestions that prognostication can improve the quality of life, even when
the probability of survival is reduced (3). Prediction represents an important aspect of patient care,
identifying high-risk patients requiring special care, and allowing low-risk patients to be reassured.
Many predictive factors have been identified that are associated with an increased risk of metastatic
disease. These have been incorporated into a prognostic algorithm, the Liverpool Uveal Melanoma
Prognostication Online (LUMPO); https://mpcetoolsforhealth.li verpool.ac.uk/matsoap/lumpo3cr_v5.htm).

This online tool was developed to estimate survival probability in patients treated for UM,
combining anatomical predictors, such as largest basal diameter of the tumour, tumour thickness,
ciliary body involvement and extra-ocular dissemination; histological predictors, including epithelioid
cell type, presence of closed loops and tumour mitotic count; and genetic predictors, including loss of
one copy of chromosome 3. The tool was validated in 2012 (4) with data from a cohort of patients
from the Liverpool Ocular Oncology Clinic (LOOC).

The first version of LUMPO was validated externally in 2015, at the Department of
Ophthalmology, University of Medical Sciences in Poznan Poland (5). The study concluded that
LUMPO is a useful tool for calculating survival probabilities in patients with uveal melanoma.
However, they emphasized that the use of cytogenetic data would make this prognosis more
accurate.

More recently, LUMPO was externally validated in the USA in 2016, in a cohort of patients
treated at the University of California, San Francisco (UCSF) (6). The results of the study revealed
that there were differences between the anatomical and clinical characteristics of the two cohorts of
patients. There were differences also in the type of treatment and lack of genetic data at the UCSF.
Despite these differences, the study showed that LUMPO can accurately estimate all-cause and
melanoma-specific mortality for patients treated at UCSF.

A new version of LUMPO (called LUMPO3) was created incorporating information on

chromosome 8q and calculating survival using competing risk methods; which objective was to
improve the model, predicting metastatic death as a specific distinctive event different from other
causes of death. LUMPO3 was internally validated using 20-fold cross validation method (7).

The aim of this study is to undertake an external validation of LUMPO3, for this, the Liverpool
Ocular Oncology Research Group (LOORG) organised a multi-centre study inviting members of the
Ophthalmic Oncology Group (OOG) to participate.

1
METHODS

Participating centres were asked to provide demographic data which included gender and
age, anatomical data including ultrasound measurements of largest basal tumour diameter, thickness,
ciliary body involvement and extraocular extension, histological data such as the presence or absence
of extravascular matrix loops, presence or absence of epithelioid cells and mitotic cell count and
finally genetic data including genetic typing based on chromosomes 3 and 8 status. Full description
on how data were obtained and classified, whether using Fluorescence in situ hybridization (Fish)
methods or Multiplex Ligation Probe Amplification (MLPA)(8) was also requested.
Survival data were collected on all patients diagnosed and / or treated for uveal melanoma
from the years 2006 to 2016, allowing adequate time to obtain sufficient follow-up information.
Anonymous data were provided by the external centres participating in the study, following their local
Institutional policies and guidelines to export patient’s data. Data were excluded if it was missing
information regarding age and sex, tumour thickness and basal tumour diameter, histologic and
genetic tumour data, or bilateral uveal melanoma (as specified previously in the spreadsheet).

The data were transferred to the data manager at the University of Liverpool (UoL), who
masked them by removing the centres identification, and actual outcomes of the UM patients before
conducting survival prediction. The data manager also assigned a pseudo-random number for each
centre, so that analysts would not know from which centre each sample was. Data were analysed
using the LUMPO3 model to predict the 10-year survival function for each patient. The survival
predictions were sent to an independent adjudicator to correlate with actuarial survival using
discrimination and calibration methods and risk stratification (7).

Subjects were stratified into prognostic groups according to their survival predictions and the
average survival prediction for each prognostic group was compared with the actuarial Kaplan-Meier
curves. Survival curves were compared visually and numerically, the model’s discrimination
performance was assessed using Harrell’s C index and the calibration was accessed graphically.
Statistical analyses were conducted using various statistic software tools. Results were presented at
the OOG group at the EVER meeting in April, 2019.

RESULTS

Baseline information

The external validation of LUMPO3 was performed on 6009 cases reported over the 10 years period
between 2006 and 2016, from patients diagnosed with uveal melanoma, obtained retrospectively from
seven external participating Centres, and from the LOOC in the UK as a comparative strategy. The
study included 4173 patients from the LOOC, 1086 from the Leiden University Medical Centre
(LUMC) in Netherlands, 218 from the Erasmus Medical Centre Hospital (EMCH) in Rotterdam, 138 from

2
the University of California-San Francisco (UCSF), 138 from the University Hospital Schleswig-Holstein
(UHSH) in Rostock, 134 from the Helmholz Institute of Eye Diseases (HIED) in Moscow, 73 from S.C. Oculistica
Oncologica (SCOO) in Italy, and 49 from the University Hospital of Essen (UHE) in Germany.

A total of 2917 were female, and 3097 patients were male. The mean age was 60.8 years.
The median basal tumour diameter was 12.88 mm, and the median ultrasound height was 7.1.
Tumour measurements were obtained….

Ciliary body involvement was absent in 4371 cases, present in 1535 cases, and unknown in 103
cases. The histologic results reported epithelioid cells present in 2294 cases and absent in 1476
cases, the mitotic counts were identified in 2809 patients, and absent in 3200, and the presence of
closed extravascular matrix loops was documented in 1032 patients and lacking in 954 patients.
Genetic data were described in …. Patients…. patients reported with chromosome 3 loss, and ….
with chromosome 8 gain.

Comparison of cohorts

There were differences noted between the cohorts of patients from the eight centres. The patients
from Moscow were younger, and their tumours were on average with a thicker basal tumour
diameter……

The prevalence of ciliary body involvement was higher in Rotterdam and Moscow, probably because
of differences in the definition of the limits of the ciliary body. It was notable in LUMC the high
percentage of patients with extraocular melanoma in relation to the number of cases in the other
groups. The number of patients with epithelioid cells also differed between the eight groups in which
the exam was performed, perhaps due to differences in data acquisition. …. patients presented
genetic data; chromosome 3 loss status was reported in …., and chromosome 8 gain in …. patients.
Patients from SCOO and EMCH presented the highest percentage of alterations in chromosome 3
and chromosome 8.

Validation of LUMPO3

To determine similarity in the study, the characteristics of the patients in the development dataset and
each of the validation data sets were visually inspected. Kaplan-Meier curves for the development
dataset and each of the validation data sets were visually inspected to determine similarity.
Discrimination and calibration were considered for each model. Discrimination refers to the ability of
the prognostic model to differentiate between those who experience the event during the study and
those who do not. The discriminative capacity of the model was measured using Harrell's c-
statistics.(9) It is measured on a scale ranging from 0.5 (no better than chance) to 1 (perfect
prognostic). Calibration refers to how closely the probability of the event predicted by the model
agrees with the observed probability.(10) Calibration was assessed graphically (10) – if predicted and
observed probabilities agree over the whole range of probabilities, the plots show a 45° line.

3
Random-effects meta-analysis was used for synthesizing Harrell’s c-statistics, while accounting for

the correlation between studies.(11) Predicted probabilities of the outcome were calculated
independently of the external validation, by the developers of the LUMPO3 model. The comparison
of patient characteristics for the development and validation datasets was represented in table with
the general characteristics of the patients, and a Kaplan-Meier curve for the outcome based on the
development and validation datasets. The data from Essen and UCSF_Old are the closest match to
the Liverpool (development) dataset in terms of survival probability.

4
1. Singh N, Bergman L, Seregard S, Singh AD. Uveal Melanoma: Epidemiologic Aspects. Second
ed. Damato B, Singh AD, editors. Germany: Springer; 2014 2014. 382 p.

2. Singh AD, Borden EC. Metastatic uveal melanoma. Ophthalmol Clin North Am.
2005;18(1):143-50, ix.

3. Damato B, Heimann H. Personalized treatment of uveal melanoma. Eye (Lond).

2013;27(2):172-9.

4. Eleuteri A, Damato B, Coupland SE, Taktak AFG. Enhancing survival prognostication in

patients with choroidal melanoma by integrating pathologic, clinical and genetic predictors of
metastasis. International Journal of Biomedical Engineering and Technology. 2012;1(1):18-35.

5. Rospond-Kubiak I, Wroblewska-Zierhoffer M, Twardosz-Pawlik H, Kociecki J. The Liverpool

Uveal Melanoma Prognosticator Online (LUMPO) for prognosing metastasis free survival in the
absence of cytogenetic data after ruthenium brachytherapy for uveal melanoma. Acta
Ophthalmologica. 2015;93:n/a-n/a.

6. DeParis SW, Taktak A, Eleuteri A, Enanoria W, Heimann H, Coupland SE, et al. External
Validation of the Liverpool Uveal Melanoma Prognosticator Online. Invest Ophthalmol Vis Sci.
2016;57(14):6116-22.

7. Eleuteri A, Taktak AF, Coupland SE, Heimann H, Kalirai H, Damato B. Prognostication of

metastatic death in uveal melanoma patients: A Markov multi-state model. Computers in Biology
and Medicine. 2018.

8. Caines R, Eleuteri A, Kalirai H, Fisher AC, Heimann H, Damato BE, et al. Cluster analysis of
multiplex ligation-dependent probe amplification data in choroidal melanoma. Mol Vis. 2015;21:1-
11.

9. Pencina MJ, D'Agostino RB. Overall C as a measure of discrimination in survival analysis:

model specific population value and confidence interval estimation. Stat Med. 2004;23(13):2109-23.

5
10. Steyerberg EW, Vickers AJ, Cook NR, Gerds T, Gonen M, Obuchowski N, et al. Assessing the
performance of prediction models: a framework for some traditional and novel measures.
Epidemiology (Cambridge, Mass). 2010;21(1):128-38.

11. Snell KIE, Hua H, Debray TPA, Ensor J, Look MP, Moons KGM, et al. Multivariate meta-
analysis of individual participant data helped externally validate the performance and
implementation of a prediction model. Journal of Clinical Epidemiology. 2016;69:40-50.