Hyperparameter Search: Grid

The grid search method generates trials on a “grid” of hyperparameter configurations and trains each trial for the number of training units specified by max_length. The user specifies a set of values for each hyperparameter via the hyperparameters field in the experiment configuration. The “grid” of hyperparameter configurations is generated by taking the product of these sets. For example, if the set of values for three separate hyperparameters aparam, bparam, and cparam are specified as {0, 1, 2}, {10, 20}, and {"c"} respectively, then the grid of tuples (aparam, bparam, cparam) generated is:

(0, 10, "c")
(0, 20, "c")
(1, 10, "c")
(1, 20, "c")
(2, 10, "c")
(2, 20, "c")

The way the set of hyperparameter values is specified depends on the type of hyperparameter:

  • const: The set of values contains just the single value. For example, cparam above could be specified as a const hyperparameter with val: c.

  • categorical: The set of values is exactly the set of categorical values. For example, bparam above could be specified as a categorical hyperparameter with vals: [10, 20].

  • int: The set of count values is taken evenly from the range [minval, maxval], inclusive of endpoints. If count is larger than the number of integer values in the range, that is interpreted as the entire range of integers in [minval, maxval]. For example, aparam above could be specified as an int hyperparameter with minval: 0, maxval: 2, and count: 3 or count: 100.

  • double: The set of count values is taken evenly from the range [minval, maxval], inclusive of endpoints. The set {0.1, 0.3, 0.5} could be specified as a double hyperparameter with minval: 0.1, maxval: 0.5, count: 3.

  • log: The set of count values is taken logarithmically evenly from the range [base^minval, base^maxval], inclusive of endpoints. For example, the set {0.00001, 0.0001, 0.001} could be specified as a log hyperparameter with base: 10, minval: -5, maxval: -3, and count: 3.

In the special case of count: 1 for int, double, or log, the midpoint (with rounding for int and with base midpoint for log) is returned.