Getting Started

Experiment Tools runs an experiment function over a parameter grid and saves all of the results. It saves with datestamps to avoid overwriting if similar tests are done multiple times. This is designed to help facilitate robust evaluation with minimal additional engineering work.

User Install

Install directly from github

pip install git+https://github.com/ml4sts/experiment-tools

Developer install

CLone and install

git clone https://github.com/ml4sts/experiment-tools
cd experiment-tools
pip install .

Basic use

The basic use is to write a function, my_expt_function that takes as input all of the parameters that you want to test and runs your code and create a dictionary with keys as each parameter and a list of the values for each parameter, param_grid and then run:

from expttools import Experiment

my_expt = Experiment(my_expt_function, param_grid)
my_expt.batch_run()

Your function should return a DataFrame of the results, it might be easiest to understand through a simple example or a more complex example of an experiment that generates datasets.