Experiment Tools#
The goal of experiment tools is to make following best practices easier for researchers doing empirical and applied machine learning work.
In the design of experiment tools we put the focus on the following :
saving results
restarting
abstracting away iterating over different options
We do so by introducing two core abstractions:
Experiment: used to run a user defined function for a combination of inputsExperimentResult: used to analyze the results
An Experiment is characterized by a user defined experiment_function and parameter_grid this
allows a researcher or practitioner to write a function that captures what they want to do and what
parameters of that procedure they want to test. These parameters are the parameters to the function and the
keys of the parameter_grid dictionary.
The Experiment object then gives some helpers to iterate over all cominations and provide structured ways to save the results.
An ExperimentResult is a data structure that helps perform common actions over the output files
produced by running a batch run of an Experiment.
Who is this tool made for?#
Diego#
Diego is a PhD student in computational biology who frequently tests different simulation models for gene expression. He needs to explore how different parameter values impact his results but finds it frustrating to set up and track all the variations manually. He wants a simple tool that lets him define parameters, run multiple experiments at once, and see the results in an organized way—without writing complex scripts.
This might be for you if…
You need to test different parameter values without manually updating code.
You want to run multiple experiments at once and compare the results easily.
You work with data analysis or simulations and need a structured way to test variations.
Rachel#
Rachel is a data analyst at an e-commerce company. She runs reports on customer trends and often tests different formulas and conditions to see which approach works best. She’s not a programmer, but she knows how to run scripts and wants a way to test different input values without manually editing code each time. She’s looking for a tool that simplifies this process so she can focus on analyzing results instead of setting up tests.
This might be for you if…
You need to test different formulas or calculations without deep programming knowledge.
You want to enter multiple values at once and see how they impact results.
You prefer a structured, automated way to compare outputs without modifying scripts.
Arjun#
Arjun is an engineer who works with manufacturing data. He needs to test different configurations for a production process, such as adjusting temperature and pressure settings, to find the most efficient combination. He doesn’t want to spend time manually changing inputs and re-running calculations—he just wants a tool where he can input his variables, let the system do the work, and get a clear output.
This might be for you if…
You work with different configurations and need a quick way to test combinations.
You want to input multiple values at once without manually modifying code.
You need structured experiment results to identify the best settings or trends.
Sofia#
Sofia is a physics professor who teaches a computational modeling course. She wants her students to explore different scientific formulas by experimenting with variables, but they often struggle with setting up the tests. She needs a tool that allows them to easily input values, run calculations, and see how the results change—without requiring them to be programming experts.
This might be for you if…
You want to experiment with different variable inputs without programming expertise.
You need to see how changes in parameters affect results in a structured format.
You are teaching or learning scientific or computational concepts and need a simple way to test equations.