Feasibility, Pilot and Proof of principle studies 1.

Feasibility studies Sometimes a full-scale study cannot be justified because there are too many unknowns, or because a critical piece of information is lacking. In such circumstances a feasibility study or a pilot study may be warranted. A feasibility study is undertaken before the real work to (i) determine whether to proceed with a project and (ii) decide the best approach to adopt. The term pilot study is sometimes used synonymously with feasibility study. However feasibility studies (in the sense we use here) correspond to what is often referred to as an external pilot . Feasibility studies can be extremely important in reducing uncertainties for planning the definitive study, and often provide reassurance to funding bodies that the work has been thoroughly thought through. A key point that is often misunderstood by investigators is that the endpoints (outcome variables) for a feasibility study are NOT the same as for the main study. Instead, the endpoints for a feasibility study are factors that affect successful trial conduct, not measures of treatment effect or safety. Questions for a feasibility study will be: What do we need to know that we currently dont in order to make the main study a success? What could go wrong in the main study? Typical areas to be addressed in a feasibility study include: Study Participants How easy are they to identify? How realistic are our eligibility criteria? How willing would they be to be randomised? How would our planned visits and assessments be scheduled? Study Investigators How suitable are our investigators in terms of qualifications, experience, probity and competing work What facilities and staff do they have at their centre? How well did they manage to recruit patients as an indicator marker of what might happen in the main study How willing are they to randomise patients when faced with the study alternatives? What is their expertise in the alternative study interventions? Estimates of key values Recruitment rates % missing data Inter-rater agreement for key outcomes costs

It is a debatable point whether potential participants should actually be randomised in a feasibility study, or whether they should be asked for hypothetical consent (which is likely to give an over-estimate). Formal sample size calculations have only a limited role in feasibility studies clearly if large, a feasibility study might just as well be a definitive one because resources will not justify both a large feasibility and a large definitive study. .However the basic justification for a feasibility study is uncertainty, so a large feasibility study is likely to take longer than necessary, waste resources in inefficient procedures, ask the wrong questions and so on. On the other hand even with a small feasibility study the answers obtained must be precise enough to be useful, and the use of confidence intervals for quantities such as primary outcome measures and recruitment rates will be helpful in judging this. 2. Internal pilot studies In contrast with a feasibility or external pilot study, an internal pilot study aims to obtain an estimate of a critical piece of information under the same conditions (procedures, eligibility criteria and outcomes) as the proposed main study. A good example is when the first 50 or so participants are used to give an estimate of the SD of the response variable, as a basis for re-estimation of sample size. Another use might be to estimate the proportion of potential subjects giving consent, which may be used to revise the total number of trial centres, or even justify a new method for recruiting participants. If such changes can be anticipated (as with a sample size re-estimate) the possibility can be written into the protocol, but otherwise a formal protocol amendment will be required,. It is therefore advisable to severely restrict the scope of an internal pilot study, and to use instead a separate feasibility study (external pilot) if the results may induce major changes to the protocol. In Nottingham CTSU we shall confine the term pilot study to mean an internal pilot. For an internal pilot, informal rules of thumb (such as the n of 10 rule) may be applied to the pilot as a way of deciding how much data should be collected before reestimating the sample size, or adjusting the number of centres, because the formal sample size estimate applies of course to the main study. However a small trial cannot be made acceptable by labelling it as a feasibility or pilot study. The Medical Research Council has now recognised that preliminary feasibility or pilot work may be needed to fully inform the development of a definitive study, and the MRCs Health Services and Public Health Research Board (HSPHRB) has recently announced a call for proposals for intervention evaluation study platforms, whose aim is to support purpose-driven, preparatory work that will underpin future definitive randomised trials, particularly those of complex, community based or public health interventions. See: http://www.mrc.ac.uk/ApplyingforaGrant/CallsForProposals/InterventionEvalu ationStudyPlatforms/index.htm

3. Proof of principle studies A proof of principle (=proof of concept) study is one which demonstrates that a particular intervention can have an effect on the condition being investigated. The outcome is often a surrogate for the real outcome (but justifiable in terms of the mechanism by which the outcome is caused). Such a study needs good adherence to the protocol and is therefore classed as an explanatory trial for which a per-protocol analysis provides the most convincing results. Such a study need not be very large but it does need to be (i) large enough to provide convincing evidence that the principle works, and (ii) the surrogate, if used, must be acceptable . Clearly therefore, a formal sample size estimate is required for such studies, although use of a surrogate may mean that the sample size can be smaller than for a definitive study.