MELvan
Capture results on the go using AI
Choose your role
MELvan
Capture results on the go using AI
Reporter
Reporter
Reviewer
Approved record
Administrator
Reporter and Reviewer access
Administrator
Overall performance for this workspace
About MELvan
An AI-assisted approach to capturing program impact
MELVan was developed collaboratively by the Precision Child and Youth Mental Health (PCYMH) Collaboratory at Children's Hospital of Eastern Ontario (CHEO) Research Institute and Vitus Consulting.
Open-source AI product architecture: Martin Mills (3den Systems)
For information on how to cite MELVan, see CITATION.cff. MELVan is licensed under the MIT License. See LICENSE.
MELvan is a wordplay on Monitoring, Evaluation, and Learning.
MELvan is an AI tool that helps program leaders capture evaluation-relevant moments as they happen — a short note, a photo, a quick observation during a meeting or in the field. MELvan then classifies each entry against a program's Results-Based Management (RBM) structure called logic model (output, outcome, ultimate goal), assigns a confidence score, and produces a short rationale.
Read the product guide in our Github repository for a high-level summary of the system and roles.
Unintended and emergent program outcomes are crucial for accountability and learning, but they're easily lost without the right tools and processes. Program leaders often notice an evaluation-relevant moment as it happens — an unintended result mentioned in a meeting, an aside in an informal conversation — but don't have time to reflect on it or document it on the spot. The moment passes, and usually so does the memory of the emergent or unintended outcome by the time an evaluator asks about it.
A leader captures a short note or photo during a meeting or daily work. MELvan classifies the entry and logs it with a confidence score and rationale. At an agreed upon cadence (e.g.: quarterly), program leadership and an external evaluation consultant review the log together, confirm or correct MELvan's classification, and feed accepted entries into the program's monitoring, learning, and evaluation work.
Traditional data collection is essential but periodic. MELvan adds a complementary layer: continuous, leader-driven capture that surfaces emergent or unintended outcomes as they unfold, rather than waiting for the next scheduled evaluation exercise. It's designed to lower the effort required to capture and classify evaluation-relevant information in daily leadership workflows, while deferring deeper interpretation to dedicated review sessions.
MELvan's knowledge base combines open-source evaluation literature with program-specific information. MELvan uses a combination of deterministic (rules-based) and non-deterministic (AI) approaches to classify reported statements against the logic model. It provides a confidence score for each classification, as well as a rationale. Classification, mapping, and their rationales are fully automated processes; the review session is where an evaluation team makes the interpretive call as to their quality. During the review, MELvan's outputs are assessed by the reviewers and accepted, modified, or rejected, and that feedback becomes an input to MELvan's learning mechanism to improve its classification model, meaning that over time it should become progressively more accurate.
A Reporter submits an entry — text, a voice recording, or a photo — and can add extra context up until it's reviewed.
MELvan summarizes the entry, classifies it as an Activity, Output, or Outcome, and proposes one or more mappings to the program's Logic Model, each with its own confidence score and rationale.
A Reviewer accepts a correct mapping, corrects it, or makes a correction right in the tool itself — nothing becomes Evidence until a Reviewer has approved at least one mapping.
An Administrator manages the program's Reporter and Reviewer accounts and its Logic Model (e.g. add a new version).
An Owner creates Administrators and can suspend accounts. Owners have read-only visibility into the workspaces (e.g. logic model or other source documents).
MELvan is assessed against five quality measures:
Traceability: every log entry carries a rationale, so the reasoning behind a classification or mapping is visible, not a black box.
Open science: the source code and supporting resources are publicly available
Resource-conscious design: constrained AI requests, transient voice processing, image normalization, temporary-file cleanup, bounded logging, and configurable infrastructure capacity all limit unnecessary computation and data retention. Environmental impact, however, isn't currently measured as we are still thinking through the best way to do this.
Most programs will benefit from being able to quickly capture program results on the go. However, for MELvan to be useful for your specific program, and for it to provide accurate classification tailored to your needs, you need to give it your specific program logic model. Therefore, you need a program logic model as a pre-condition to use MELvan.
If your program logic model isn't built yet: that's your starting point.
If you already have a logic model: contact Roxana Salehi (*roxana@vitus.ca)* to request access to the demo version in order to assess MELvan's suitability for your purposes. Access is typically given for 2-4 weeks.
Newly created administrator accounts can configure the logic model for their workspace — they have the choice to either clone the 'demo' logic model, a generic logic model for an imaginary mental health program for adults, or they can create their own logic model which must comply with the required structure.
If you want to use MELvan on an ongoing basis, then we recommend you discuss its use with your organization and ensure it complies with any existing responsible AI frameworks in your organization.
If your technology team is able to support installation of cloud-based applications for Azure (currently, Azure is the only supported Cloud provider), you are invited to fork our open access GitHub to install your own version on your own Azure instance.
Otherwise, contact Vitus Consulting if (a) you require professional services to install MELvan on your own infrastructure, or (b) if your organization has determined that it can use a third-party (i.e. Vitus's own) version of MELvan. Consulting and/or subscription fees will apply.
Yes.