Package: cvms 1.6.2.9000

Ludvig Renbo Olsen

cvms: Cross-Validation for Model Selection

Cross-validate one or multiple regression and classification models and get relevant evaluation metrics in a tidy format. Validate the best model on a test set and compare it to a baseline evaluation. Alternatively, evaluate predictions from an external model. Currently supports regression and classification (binary and multiclass). Described in chp. 5 of Jeyaraman, B. P., Olsen, L. R., & Wambugu M. (2019, ISBN: 9781838550134).

Authors:Ludvig Renbo Olsen [aut, cre], Hugh Benjamin Zachariae [aut], Indrajeet Patil [ctb], Daniel Lüdecke [ctb]

cvms_1.6.2.9000.tar.gz
cvms_1.6.2.9000.zip(r-4.5)cvms_1.6.2.9000.zip(r-4.4)cvms_1.6.2.9000.zip(r-4.3)
cvms_1.6.2.9000.tgz(r-4.4-any)cvms_1.6.2.9000.tgz(r-4.3-any)
cvms_1.6.2.9000.tar.gz(r-4.5-noble)cvms_1.6.2.9000.tar.gz(r-4.4-noble)
cvms_1.6.2.9000.tgz(r-4.4-emscripten)cvms_1.6.2.9000.tgz(r-4.3-emscripten)
cvms.pdf |cvms.html✨
cvms/json (API)
NEWS

# Install 'cvms' in R:
install.packages('cvms', repos = c('https://ludvigolsen.r-universe.dev', 'https://cloud.r-project.org'))

Peer review:

Bug tracker:https://github.com/ludvigolsen/cvms/issues

Datasets:

On CRAN:

33 exports 37 stars 3.38 score 83 dependencies 4 dependents 1 mentions 468 scripts 1.8k downloads

Last updated 2 months agofrom:1cba75046f. Checks:OK: 7. Indexed: yes.

TargetResultDate
Doc / VignettesOKAug 30 2024
R-4.5-winOKAug 30 2024
R-4.5-linuxOKAug 30 2024
R-4.4-winOKAug 30 2024
R-4.4-macOKAug 30 2024
R-4.3-winOKAug 30 2024
R-4.3-macOKAug 30 2024

Exports:baselinebaseline_binomialbaseline_gaussianbaseline_multinomialbinomial_metricscombine_predictorsconfusion_matrixcross_validatecross_validate_fnevaluateevaluate_residualsfontgaussian_metricsmodel_functionsmost_challengingmulticlass_probability_tibblemultinomial_metricsplot_confusion_matrixplot_metric_densitypredict_functionspreprocess_functionsprocess_info_binomialprocess_info_gaussianprocess_info_multinomialreconstruct_formulasselect_definitionsselect_metricssimplify_formulasum_tile_settingssummarize_metricsupdate_hyperparametersvalidatevalidate_fn

Dependencies:backportsbayestestRbootcheckmateclasscliclockcodetoolscolorspacecpp11data.tabledatawizarddiagramdigestdplyrfansifarverfuturefuture.applygenericsggplot2globalsgluegowergroupdata2gtablehardhatinsightipredisobandKernSmoothlabelinglatticelavalifecyclelistenvlme4lubridatemagrittrMASSMatrixmgcvminqaMuMInmunsellnlmenloptrnnetnumbersnumDerivparallellyparameterspillarpkgconfigplyrpROCprodlimprogressrpurrrR6RColorBrewerRcppRcppEigenrearrrrecipesrlangrpartscalesshapeSQUAREMstringistringrsurvivaltibbletidyrtidyselecttimechangetimeDatetzdbutf8vctrsviridisLitewithr

The available metrics in cvms

Ludvig Renbo Olsen

Rendered fromavailable_metrics.Rmdusingknitr::rmarkdownon Aug 30 2024.

Last update: 2023-06-04
Started: 2020-04-13

Creating a confusion matrix with cvms

Ludvig Renbo Olsen

Rendered fromCreating_a_confusion_matrix.Rmdusingknitr::rmarkdownon Aug 30 2024.

Last update: 2023-06-04
Started: 2020-04-13

Cross-validating custom model functions with cvms

Ludvig Renbo Olsen

Rendered fromcross_validating_custom_functions.Rmdusingknitr::rmarkdownon Aug 30 2024.

Last update: 2023-06-04
Started: 2020-04-13

Evaluate by ID/group

Ludvig Renbo Olsen

Rendered fromevaluate_by_id.Rmdusingknitr::rmarkdownon Aug 30 2024.

Last update: 2023-06-04
Started: 2020-04-13

Multiple-k: Picking the number of folds for cross-validation

Ludvig Renbo Olsen

Rendered frompicking_the_number_of_folds_for_cross-validation.Rmdusingknitr::rmarkdownon Aug 30 2024.

Last update: 2023-06-04
Started: 2021-06-06

Readme and manuals

Help Manual

Help pageTopics
cvms: A package for cross-validating regression and classification modelscvms-package cvms
Create baseline evaluationsbaseline
Create baseline evaluations for binary classificationbaseline_binomial
Create baseline evaluations for regression modelsbaseline_gaussian
Create baseline evaluationsbaseline_multinomial
Select metrics for binomial evaluationbinomial_metrics
Generate model formulas by combining predictorscombine_predictors generate_formulas
Compatible formula termscompatible.formula.terms
Create a confusion matrixconfusion_matrix
Cross-validate regression models for model selectioncross_validate
Cross-validate custom model functions for model selectioncross_validate_fn
Evaluate your model's performanceevaluate
Evaluate residuals from a regression taskevaluate_residuals
Create a list of font settings for plotsfont
Select metrics for Gaussian evaluationgaussian_metrics
Examples of model_fn functionsmodel_functions
Find the data points that were hardest to predicthardest most_challenging
Generate a multiclass probability tibblemulticlass_probability_tibble
Select metrics for multinomial evaluationmultinomial_metrics
Musician groupsmusicians
Participant scoresparticipant.scores
Plot a confusion matrixplot_confusion_matrix
Density plot for a metricplot_metric_density
Precomputed formulasprecomputed.formulas
Examples of predict_fn functionspredict_functions
Predicted musician groupspredicted.musicians
Examples of preprocess_fn functionspreprocess_functions
A set of process information object constructorsas.character.process_info_binomial as.character.process_info_gaussian as.character.process_info_multinomial print.process_info_binomial print.process_info_gaussian print.process_info_multinomial process_info_binomial process_info_gaussian process_info_multinomial
Reconstruct model formulas from results tibblesreconstruct_formulas
Select model definition columnsselect_definitions
Select columns with evaluation metrics and model definitionsselect_metrics
Simplify formula with inline functionssimplify_formula
Create a list of settings for the sum tiles in plot_confusion_matrix()sum_tile_settings
Summarize metrics with common descriptorssummarize_metrics
Check and update hyperparametersupdate_hyperparameters
Validate regression models on a test setvalidate
Validate a custom model function on a test setvalidate_fn
Wine varietieswines