Videos

Five examples, each from a real study: what is predicted, what kind of model it is and which engine wins.

The videos are real recordings of the Studio running a real study; the voice is synthetic and in English, and they have subtitles.

• BinaryLength 0:55

  Is this tree diseased?

  Satellite imagery · binary classification

  Each row is a group of pixels from a satellite image (4,839 rows, five numeric columns), and the model says whether they are diseased trees or not, with its probability. Several engines compete on the same data and the MatrixAI TabM neural network wins: AUROC 0.999 on the held-out test, which no engine saw while training. Then the model is exported as a Docker service and, in a real terminal, it is built, started and answers a row.

  "Wilt" dataset: Brian Johnson (2013), UCI Machine Learning Repository, https://doi.org/10.24432/C5KS4M, licensed under CC BY 4.0. Adapted by OpenML (id 40983, v2): variables renamed to match the description; no changes to the data itself.

• MulticlassLength 0:53

  Which digit was written by hand?

  Handwriting on a tablet · multiclass classification

  10,992 digits handwritten on a tablet, each described by 16 numbers, and the model says which of the ten it is (0 to 9), with the probability of each. It is a multiclass classification. The MatrixAI TabM neural network wins, with accuracy 0.992 on the held-out test, and the Studio says its improvement over the runner-up is demonstrated.

  "Pen-Based Recognition of Handwritten Digits" (pendigits) dataset: E. Alpaydin and Fevzi Alimoglu, UCI Machine Learning Repository, https://doi.org/10.24432/C5MG6K, licensed under CC BY 4.0. Available through OpenML (id 32, v1).

• BinaryLength 0:52

  Will this climate simulation crash?

  Climate models · binary classification

  540 runs of a climate model, each with the 18 parameters it was launched with, and the model says whether it will crash or not, with its probability. It is a binary classification. Several engines compete and LightGBM, a tree model, wins: AUROC 0.963 on the held-out test.

  "Climate Model Simulation Crashes" dataset by D. Lucas, R. Klein, J. Tannahill, D. Ivanova, S. Brandon, D. Domyancic and Y. Zhang (2013), UCI Machine Learning Repository, https://doi.org/10.24432/C5HG71, licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Adapted by OpenML (id 40994, v4): the first two variables (the experiment batch) are removed and the target becomes a category.

• RegressionLength 0:53

  How much violent crime will a community have?

  US communities data · regression

  1,994 US communities with 126 socio-economic columns, and the model predicts a number: the violent-crime rate per inhabitant, on a 0-to-1 scale. It is a regression. The neural network had the best mean, but without enough margin: it is a tie, and the Studio delivers the tree, LightGBM, the simpler model. Its typical error (RMSE) on the held-out test is 0.127.

  "Communities and Crime" dataset by Michael Redmond (La Salle University), UCI Machine Learning Repository, https://doi.org/10.24432/C53W3X, licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Adapted by OpenML (id 42730, v2): the `communityname` attribute has been removed.

• BinaryLength 0:52

  Will this dress be recommended?

  Fashion sales · the tie-break

  500 dresses with 12 attributes (style, price, season…), and the model says whether each one will be recommended, based on its sales. It is a binary classification, and it shows the tie-break: the neural network had the best mean, but its advantage over the best tree is within the noise, so the Studio delivers the tree (HGB), the simpler model. It is a hard dataset and the figure is shown as it is: AUROC 0.578 on the held-out test.

  "Dresses_Attribute_Sales" dataset by Muhammad Usman and Adeel Ahmed (2014), UCI Machine Learning Repository, https://doi.org/10.24432/C56W3V, licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Adapted by OpenML (id 23381): 500 rows vs. 501 in the current UCI listing (unexplained difference).