Real examples · they open without internet

Living Reports of AI models generated with MatrixAI Studio

A Living Report is a single .html file that opens offline: it carries the model and everything needed to read it, and it needs no server. These ones were made with MatrixAI Studio on public datasets, so you can see how a model is explained before using it on your own data.

What it carries

  • The model card: what data it was made with, how it was measured and which engine won.
  • A practical reading at the top, with its uncertainty: in a binary problem, how many cases it detects and how many false alarms it gives; in regression, how far off it is on average, in the units of what it predicts; with several classes, how much it gets right in each one.
  • The curves of the test, by task: binary, the ROC curve, the precision and sensitivity curve and the reliability; regression, predicted against real in bins; several classes, the confusion matrix and the reliability of the chosen class.
  • Only in a binary problem: the model’s decision map and the automation simulator, where you move two thresholds and see what share the AI would decide alone and how many mistakes it would make. In regression and with several classes those two tabs say “does not apply” and why.
  • Try and explain a row: type the values of a case and the report itself computes the model’s answer and explains it (in regression, with its interval when the study has one and a warning if the case is out of range).

What it does not carry or say

  • No row of the test: it carries the model and aggregate results, not the data it was measured on.
  • It is not a clinical tool and not a decision: it is an example of how a model is explained, in health, industry or risk alike.
  • The score is that of a public dataset. It does not say how the model would work on your data: that is what the Studio is for, which measures on yours.

The examples

Each card carries what the report says about itself. “View here” opens it in an isolated frame inside this page; on a phone “Open full screen” usually works better.

  • Binary classification

    Climate model simulation crashes

    Winning engine
    LightGBM
    Score
    AUROC 0.964 on the held-out test (108 rows)
    Size
    443.5 KB
    Signed with
    «MatrixAI · ejemplos» · 7d82f59e…ec4a25ca

    Data: Climate Model Simulation Crashes. Licence of the data: CC BY 4.0.

    Citation: "Climate Model Simulation Crashes" dataset by D. Lucas, R. Klein, J. Tannahill, D. Ivanova, S. Brandon, D. Domyancic and Y. Zhang (2013), UCI Machine Learning Repository, https://doi.org/10.24432/C5HG71 doi.org/10.24432/C5HG71

    Changes: Adapted by OpenML (id 40994, v4): the target becomes a category, and the first two variables (the experiment batch, "Study" and "Run") are marked to be ignored; here they are removed. MatrixAI trained and measured a model on the data.

    Open full screenDownloadCheck its signature

    Download it, drop it into “Check a signature” and paste the fingerprint of the key shown above.

  • Binary classification

    Defects in satellite flight software (pc1)

    Winning engine
    LightGBM
    Score
    AUROC 0.793 on the held-out test (221 rows)
    Size
    443.5 KB
    Signed with
    «MatrixAI · ejemplos» · 7d82f59e…ec4a25ca

    Data: pc1. Licence of the data: CC BY 4.0.

    Citation: "pc1" dataset (NASA Metrics Data Program / PROMISE) by Tim Menzies (NC State University), Zenodo, https://doi.org/10.5281/zenodo.268456 doi.org/10.5281/zenodo.268456

    Changes: MatrixAI's catalogue records no removed column and no changed value in this dataset. MatrixAI trained and measured a model on the data.

    Open full screenDownloadCheck its signature

    Download it, drop it into “Check a signature” and paste the fingerprint of the key shown above.

  • Regression

    Violent crime in US communities

    Winning engine
    LightGBM
    Score
    RMSE 0.13 on the held-out test (399 rows)
    Size
    513.9 KB
    Signed with
    «MatrixAI · ejemplos» · 7d82f59e…ec4a25ca

    Data: Communities and Crime. Licence of the data: CC BY 4.0.

    Citation: "Communities and Crime" dataset by Michael Redmond (La Salle University), UCI Machine Learning Repository, https://doi.org/10.24432/C53W3X doi.org/10.24432/C53W3X

    Changes: Adapted by OpenML (id 42730, v2): the communityname attribute is marked to be ignored. UCI also does not count state, county, community and fold among the predictive columns (its listing: "122 predictive, 5 non-predictive, 1 goal"). Here all five are removed. MatrixAI trained and measured a model on the data.

    Open full screenDownloadCheck its signature

    Download it, drop it into “Check a signature” and paste the fingerprint of the key shown above.

  • Binary classification

    Breast masses: malignant or benign (WDBC)

    Winning engine
    LightGBM
    Score
    AUROC 0.999 on the held-out test (113 rows)
    Size
    453.9 KB
    Signed with
    «MatrixAI · ejemplos» · 7d82f59e…ec4a25ca

    Data: Breast Cancer Wisconsin (Diagnostic). Licence of the data: CC BY 4.0.

    Citation: "Breast Cancer Wisconsin (Diagnostic)" dataset by William Wolberg, Olvi Mangasarian, Nick Street and W. Street (1993), UCI Machine Learning Repository, https://doi.org/10.24432/C5DW2B. Introductory paper according to UCI: W. N. Street, W. H. Wolberg and O. L. Mangasarian (1993), "Nuclear feature extraction for breast tumor diagnosis", Electronic Imaging, https://doi.org/10.1117/12.148698. doi.org/10.24432/C5DW2B

    Changes: Adapted: the ID column (a record number) is removed, the diagnosis "M"/"B" becomes "Maligno"/"Benigno" and the 30 measurements are renamed (media_, error_tipico_ and peor_ plus the name of the measurement); every other value is the one in UCI's file, unmodified. It is not a MatrixAI Phase 0 dataset, nor a diagnostic tool. MatrixAI trained and measured a model on the data.

    Open full screenDownloadCheck its signature

    Download it, drop it into “Check a signature” and paste the fingerprint of the key shown above.

It is signed: how to check it

Each report carries an Ed25519 signature made with a MatrixAI key created only for these examples. Its fingerprint is this:

Fingerprint of the key “MatrixAI · ejemplos” (SHA-256)

7d82f59e d8b433a6 ab15c33f c099b7df 64ac4790 577cd87d 310ab197 ec4a25ca

To check a report

  1. Download the report with “Download”.
  2. Open “Check a signature” and drop the file there.
  3. Choose “Just its fingerprint” and paste the fingerprint above.
  4. Press “Check the signature”. If it says “Valid signature”, the file is the one signed with that key and nobody has touched it since.

Check a signature

A valid signature does not say the model is correct, or when it was signed. It says the file has not changed since whoever holds that key signed it.

Compare it through another channel than this website too: a fingerprint copied from the same place as the file adds no trust.

Licences

Each report is a work of MatrixAI Studio. The code of its evaluator is inside the file itself, readable and under AGPL-3.0, with its SPDX header.

The data of each example belongs to third parties and is used under their licence: the full citation, the link and the changes we made are on its card.