Outcome measurement in Orthopaedics has had increasing importance over the past few years. This trend has long been present in the US and in some European countries (e.g. Belgium and the Netherlands). An explanation for its rising importance is the switch from fee-per-service to a value-based healthcare system where a positive impact is clearly objectified for a particular operation.

“On the long run, tracking patient assessment of pain and function can provide insights into the effectiveness of hip and knee arthroplasty across a much broader patient population rather than the relatively small number of patients that suffer implant failures and require surgery. Perhaps most importantly, PRO data reflect the patient’s perspective on the outcome of the surgery – described as “the one-on-one result of our physicians,” by one orthopedic surgeon.

Why should we measure outcomes?

To increase quality

A Clinician’s central goal is to help his patients cure their illnesses, ease their discomfort, and help them manage their health over time. Outcome data is required in order to achieve this efficiently and meticulously.

Delivering and demonstrating high-quality care in a cost-efficient way is necessary to measure the impact of certain decisions. Both clinical and patient-reported outcome measurement allows the evaluation of certain processes, results of operations, structures, etc.

To demonstrate superior outcomes

Today, hospital rankings are based on basic clinical indicators, such as mortality and infection rates, as well as on reputation. We believe these indicators do not capture the full picture since what matters most are the outcomes that patients experience.

Provable quality is associated with the measurement and publication of various factors. Orthopride is a good example of where you can have an objective revision as the endpoint. This register, in addition to a wide coverage, is perfect for traceability and epidemiology but contains few data points to justify clinical benchmarking clearly (outcomes, comorbidities, etc).

To get prepared for the shift to value-based payment

Payers want to know that they are paying for the right treatments and that care is being delivered efficiently. Therefore, they also offer the greatest value.

Value-based healthcare with an adjusted reimbursement is still too slowly being implemented, but it seems wise for an orthopedic department to be in control and to ensure that a correct outcome measurement system is implemented.

Case-study: which PATIENT REPORTED outcomes?


From the benchmarking of different sources (European / US), there is a unique constant, we should always measure HRQL in combination with a disease-specific. Here are some of the validity conditions:

  • A validated version in the language of choice: a validated questionnaire for example in the Dutch language.
  • User-friendly/limited size: a short list gives less burden for the patient and for the person who analyzes, therefore, less than 10 questions is considered user-friendly.
  • Connection with foreign orthopedic registrations: in order to be able to compare between countries in the future, priority should be given to an instrument that is also internationally used in registrations.
  • Future-proof: a PROM must be able to measure effects in a varied group of patients with the same condition. A PROM with a ceiling effect should therefore be avoided.
  • Informative: a PROM must provide information about the outcome of the treatment, both preoperatively about the indication and a difference measurement (pre / post) about the effect of the treatment.
  • No license fees: The starting point is to recommend free questionnaires.

How  to set-up a PRO-system

It is important to do this in coordination with an umbrella association such as ICHOM, clinical registries atc. . Here are some listed steps, based on the American Joint Replacement Society that can ensure that a clear system can be set up:

1. By defining a PRO team:

  • Fully internal by a clinical research nurse (admission, follow-up hit rate, analysis, etc.) or through a partnership with a health IT company, such as LynxCare
  • Remote capture PRO
  • Kiosk on the service with touchscreen
  • Email / SMS reminder for patients to optimize compliance
  • Admission of patients is automatic
  • Direct combination with clinical outcome measures

2. By defining goals for PRO program:

  • Clinical study with a specific goal?
  • Do we want to set up a competitive benchmark with peers?
  • Do we want to quantify outcomes from the patient’s perspective?
  • Do we want to look at increased functionality or reduced pain postop?
  • Are there other critical parameters regarding health-related – patient reported outcomes that we must follow?
  • Do we want to capture referrals for pain management?
  • Do we want to identify patients who are at high risk of poor outcomes (clinical + patient reported)
  • Do we want to measure the overall improvement in patient health?
  • Do we want to use the PROs in practice during the consultation?
  • Do we want to use PROs to evaluate qualitative optimization initiatives?
  • Do we want to make it possible to compare both procedures and operation protocols with regard to outcomes?

3. By determining the appropriate PRO measure:

1. Health-related questionnaires (Example joint replacement):

  • VR-12: http://www.bu.edu/sph/research/research-landing-page/vr-36-vr-12-and-vr-6d/
  • PROMIS-10 Global: www.healthmeasures.net
  • SF-12: http://www.rand.org/health/surveys_tools/mos/mos_core_12item.html
  • SF-36: http://www.rand.org/health/surveys_tools/mos/mos_core_36item.html
  • EQ-5D: http://www.euroqol.org/

2. Procedure-specific questionnaires ( Example joint replacement):

  • HOOS, JR .: https://www.hss.edu/hoos-jr-koos-jr-outcomes-surveys.asp
  • KOOS, JR .: https://www.hss.edu/hoos-jr-koos-jr-outcomes-surveys.asp
  • HOOS: http://www.koos.nu/
  • KOOS: http://www.koos.nu/
  • Oxford Hip: http://isis-innovation.com/outcome-measures/oxford-hip-score-ohs/
  • Oxford Knee: http://isis-innovation.com/outcome-measures/oxford-knee-score-oks/
  • KSS: http://www.kneesociety.org/web/outcomes.html
  • Harris Hip Score: http://www.orthopaedicscore.com/
  • WOMAC: http://www.womac.org/
Ask the right questions when setting up a PRO-system:
  • What is the questionnaire about – QALY or function of the patient?
  • Do we want to understand both better or only focus on one aspect?
  • What is the associated hurdle for the patient?
  • Length of the questionnaire?
  • How much FTE do you need internally (or no if you go through a partner)
  • Who will register the patients + follow up questionnaires?
  • What is the cost (licensing) of using such questionnaires?
  • Which translations are appropriate for our patient population?

4. With the right timing:

  • How often would we like to be a PROM post-surgery?
  • At what time points do our patients consistently come back for a follow-up visit? Is this the best time to capture the PROMs? (If patient does not complete via internet prior to visit, we have a chance to complete a visit to the clinic.
  • What time frame is needed to address our primary objectives for PRO data collection (e.g., data at any clinic visit. pre-operative and one year data to measure minimum reporting requirements for payers)?
  • How many time points for data collection can be managed by our staff?
  • How are you patient willing to complete surveys? Should a random subgroup be utilized for more frequent assessment (this may be an option for research driven initiatives)?
  • What is our hospital status regarding federal quality initiatives? Are we in one of the CJR geographic areas?

5. By developing workflow:

  • What does our current EHR platform provide? Is the infrastructure sufficient for data capture? Do our clinics or affiliated practice groups share the same EHR?
  • What do other orthopedic charting vendors offer in regards to PROs cf LynxCare?
  • What are your patient population characteristics? Will they have access to complete the forms electronically?
  • What follow-up time points will be the most manageable to capture in clinic? Or do we only want to collect off site via a secure patient portal? Or both?
  • Will there be adequate time set aside for clinical staff to manage the surveys, track patient enrollment, monitor data completion quality and rates, and follow up on problems or will we use a third party for the collection?
  • How do we frame the PRO discussion with patient?


When researching, it is clear that a PROMS system has already been set up in several countries. Below are the main highlights regarding the important points that should be filled before setting up a clear guideline.

  • Optimum workflow vs administrative workload: often lacks operational efficiency when picking up PROMS. These PROs are best captured in an outside hospital setting to not burden the service with administrative workload. If it is not possible, for example due to the patient’s age, there should always be the possibility to do this on the service, with LynxCare for example. Make sure to process sufficient clinical outcome data with the PROs to use the database for multiple purposes.
  • Golden standard:  minimize health related issues and diseases and establish a specific procedure combination.
  • Frequency: correctly capture postoperative timing.
  • Initially intend to involve a third party who will monitor and standardize this IT-moderately between various test centers.
  • Analysis of the data: take Ceiling-effect with you. Define positive and negative outliers (centers) and define CASE MIX adjustments (age, gender, combination) to standardize intra-inter hospital as much as possible.