Implementing Overview

Full scale implementation of the digital health solution in real-world settings based on what has been planned occurs during this phase. This includes implementation of the digital solution, which may be staggered to allow for incremental learning, feedback cycles and modification of the technology, the team involved, or the clinical or administrative routines.  The implementation phase should also include embedded metrics to inform iterative improvements. An evaluation conducted in the Implementing phase generally seeks to answer these kinds of questions:

  • What does implementation rollout look like in real-world settings?
  • What (if any) changes (intended and un-intended) have occurred as a result of implementation?
  • What have we learnt so far and how can we revise policy, practice or implementation in response to these lessons?
  • What are the short (6months-2years), medium (2-4years) or long-term (>4years) outcomes of the digital health solution?

Implementing Matrices

Macro

This involves stakeholders at the federal, provincial and territorial levels of governance and includes governmental funding agencies and pan-Canadian health organizations. At this level, policies and regulations are set that influence the meso and micro levels.

Meso

This includes regional or subregional health authorities and organizations, as well as primary care networks. At this level, direct investments and decisions are made to deliver healthcare services to entities within specific service areas. Decisions at this level may ultimately influence the micro level.

Micro

At the institutional-level, hospitals, clinics, and community care centres are the main spheres of influence here. At this level, the health impacts of digital health investments are directly experienced by providers and beneficiaries of service delivery and are influenced by institutional policies and administrative procedures.

Click on each heading below to learn more about how each construct in the implementing matrix functions at the macro, meso, and micro levels:

  • Equity

    Health equity implies that everyone has a fair opportunity to attain long and healthy lives and that no one is disadvantaged from achieving this potential irrespective of their social, economic, geographic, demographic, racial or ethnic grouping.

    Digital Health Equity is achieved when all people have equal opportunity to access, use and benefit from digital health to attain long and healthy lives.

  • Sustainability

    This is the process by which the digital health solution and the practices and policies that support it become institutionalized or integrated within the structures and systems for health care delivery.

  • Contextual factors

    This encompasses the wider institutional, sociocultural and economic environment of digital health implementation at the macro, meso, and micro levels that can act to enable or constrain implementation.

  • Acceptability

    Acceptability is “a multi-faceted construct that reflects the extent to which people delivering or receiving healthcare using a digital health solution consider it to be appropriate based on anticipated or experienced cognitive and emotional responses to the intervention”.

  • Appropriateness

    This is related to the fit, relevance, and compatibility of the digital health solution for a given setting, provider or patient, to address a specific health condition. Digital health solutions should be clinically relevant and tailored to align with the comfort, needs and preferences of target end user and beneficiaries.

  • Efficiency

    Efficiency refers to optimal use of available resources (financial & human) to achieve set goals of a digital health solution.

    How well does a digital health solution improve efficiency of health care delivery and workforce productivity?

    It can also refer to how well a digital health solution or tool is designed. Poor user interface design can reduce efficiencies increasing the likelihood of data or inputting errors. (E.g., too many menus, alerts, high proficiency required to use.)

  • Effectiveness

    Effectiveness is the performance of a digital health solution in the real world, including in routine clinical practice, home or community care, and in the context of continuity of care with individuals or local dispersed teams and referral services.

  • Fidelity

    This is the degree to which a digital health solution is implemented as prescribed during the planning stage and delivered as intended. It also includes adaptations made to the technology and changes to the implementation strategy.