API Reference

class benchit.BenchmarkObj(df_timings, dtype='timings', multivar=False, multiindex=False)

Bases: object

Class that holds various methods to benchmark solutions on various aspects of benchmarking metrics. This also includes timing and plotting methods. The basic building block is a pandas dataframe that lists timings off various methods. The index has the various datasets and headers are functions. This class is intended to hold timings data. It is the central building block to benchmarking workflow..

copy()

Make a copy.

Returns:Copy of input BenchmarkObj object.
Return type:BenchmarkObj
drop(labels, axis=1)

Drop functions or datasets off the benchmarking object based on column or index values. It is an in-place operation.

Parameters:labels (Any scalar or list or tuple of scalars) – Column or index value(s) to be dropped.
Returns:NA.
Return type:None
plot(set_xticks_from_index=True, xlabel=None, ylabel=None, colormap='jet', logx=False, logy=None, grid=True, linewidth=2, rot=None, dpi=None, fontsize=14, specs_fontsize=None, tick_fontsize=None, label_fontsize=None, legend_fontsize=None, figsize=None, specs_position='left', debug_plotfs=False, pause_timefs=0.1, modules=None, use_frame=False, sp_argID=0, sp_ncols=-1, sp_sharey=None, sp_title_position='center', sp_title_fontsize=None, sp_show_specs=True, save=None, **kwargs)

Plot dataframe using given input parameters.

Parameters:

  • set_xticks_from_index (bool, optional) – Flag to use dataframe’s index to set set_xticklabels or not.

  • xlabel (str, optional) – Xlabel string.

  • ylabel (str, optional) – Ylabel string.

  • colormap (str, optional) – String that decides the colormap for plotting

  • logx (bool, optional) – Flag to set x-axis scale as log or linear.

  • logy (None or bool, optional) – If set as None, it detects default boolean flag using input Object datatype to be used as logy argument for plotting that decides the y-axis scale. With True and False, the scale is log and linear respectively. If set as boolean, it is used directly as logy argument.

  • grid (bool, optional) – Flag to show grid or not.

  • linewidth (int, optional) – Width of line to be used for plotting.

  • rot (int or None, optional) – Rotation for ticks (xticks for vertical, yticks for horizontal plots).

  • dpi (float or None, optional) – The resolution of the figure in dots-per-inch.

  • fontsize (float or int or None, optional) – Fontsize used across specs_fontsize, tick_fontsize and label_fontsize if they are not set.

  • specs_fontsize (float or int or None, optional) – Fontsize for specifications text displayed as title.

  • tick_fontsize (float or int or None, optional) – Fontsize for xticks and yticks.

  • label_fontsize (float or int or None, optional) – Fontsize for xlabel and ylabel.

  • figsize (tuple of two integers or None, optional) – Tuple with syntax (figure_width, figure_height) for the figure window. This is applied only for environemnts where full-screen viewing is not possible.

  • specs_position (None or str, optional) – str that decides where to print specs information. Options are : None(default), ‘left’, ‘right’ and ‘center’.

  • debug_plotfs (bool, optional) – Flag to decide whether to display debug info on fullscreen showing of plot. This is used only for interactive backends.

  • pause_timefs (float, optional) – This is a pause number in seconds, used for plot to be rendered in fullscreen before saving it.

  • modules (dict, optional) – Dictionary of modules.

  • use_frame (bool, optional) – This indicates whether to use a frame or not. For subplot, this applies a frame to each subplot.

  • sp_argID (int, optional) – This is specific to subplot case, when applicable (combinations are possible). This represents argument index for the input datasets to be used as the base (for x-axis labelling). This is based on 0-based indexing. Default argument index is 0, i.e. the first argument.

  • sp_ncols (int, optional) – This is specific to subplot case, when applicable (combinations are possible). This denotes the number of columns used to create subplot grid.

  • sp_sharey (str or None, optional) – This is specific to subplot case, when applicable (combinations are possible). This is used to indicate if and how the y-values are to be shared. Accepted values and their respective functionalities are listed below :

    None : y-values are not shared. ‘row’ or ‘r’: y-values are shared among same row of subplots. ‘global’ or ‘g’: y-values are shared globally across all subplots.

  • sp_title_position (str, optional) – This is specific to subplot case, when applicable (combinations are possible). This indicates where to place the title for each subplot. Available values are - ‘left’, ‘center’ or ‘right’ respective to their positions.

  • sp_title_fontsize (float or int or None, optional) – This is specific to subplot case, when applicable (combinations are possible). Fontsize for title for subplots that shows the grouping argument(s).

  • sp_show_specs (bool, optional) – This decides whether to show specifications or not. Default is True, i.e show specifications.

  • save (str or None, optional) – Path to save plot.

  • **kwargs – Options to pass to pandas plot method, including kwargs for matplotlib plotting method.
Returns:

Plot of data from object’s dataframe.
Return type:

matplotlib.axes._subplots.AxesSubplot

Notes

All subplot specific arguments have prefix of “sp_”.

props()

Show object properties without the dataframe.

Parameters:None
Returns:NA
Return type:None
rank(mode='range')

Rank different functions based on their performance number and rank them by changing the columns order accordingly. It is an in-place operation.

Parameters:mode (str, optional) – Sets the ranking criteria to rank different functions. It must be one among - ‘range’, ‘constant’, ‘index’.
Returns:NA.
Return type:None
reset_columns()

Reset columns to original order.

scaled_timings(ref)

Evaluate scaled timings for all function calls with respect to one among them.

Parameters:ref (int or str or function) – Input value represents one of the headers in the input BenchmarkObj. The scaled timings for all function calls are computed with respect to this reference.
Returns:Scaled timings.
Return type:BenchmarkObj
show_columns()

Get reference to inherent dataframe columns.

Parameters:None
Returns:Array of inherent dataframe columns.
Return type:pandas.core.indexes.base.Index
show_index()

Get reference to inherent dataframe columns.

Parameters:None
Returns:Array of inherent dataframe columns.
Return type:pandas.core.indexes.base.Index
speedups(ref)

Evaluate speedups for all function calls with respect to one among them.

Parameters:ref (int or str or function) – Same as with scaled_timings.
Returns:Speedups.
Return type:BenchmarkObj
to_dataframe(copy=False)

Return underlying pandas dataframe object.

benchit.bench(df, dtype='t', copy=False, multivar=False, multiindex=False)

Constructor function for creating BenchmarkObj object from a pandas dataframe. With input arguments, it could set as a timings or speedups or scaled-timings object. Additionally, the dataframe could be copied so that source dataframe stays unaffected.

Parameters:
  • df (pandas dataframe) – Dataframe listing the timings or speedups or scaled-timings or just any 2D data, i.e. number of levels with rows and columns is 1. Also, the dataframe should have the benchmarking information setup in the standardized setup way. Columns represent function names, alongwith df.columns.name assigned as ‘Functions’. Index values represent dataset IDs, alongwith df.index.name assigned based on dataset type.
  • dtype (str, optional) – Datatype value that decides between timings or speedups or scaled-timings. Mapping strings are : ‘t’ -> timings, ‘st’ -> scaled-timings, ‘s’ -> speedups.
  • copy (bool, optional) – Decides whether to copy data when constructing benchamrking object.
Returns:

Data stored in BenchmarkObj.
Return type:

BenchmarkObj
benchit.timings(funcs, inputs=None, multivar=False, input_name=None, indexby='auto')

Evaluate function calls on given input(s) to compute the timing. Puts out a dataframe-like object with the input properties being put into the header and index names and values.

Parameters:
  • funcs (list or tuple) – Contains the functions to be timed.
  • inputs (list or tuple or None, optional) – Each elements of it represents one dataset each.
  • multivar (bool, optional) – Decides whether to consider single or multiple variable input for feeding into the functions. As such it expects all functions to accept inputs in the same format. With the value as False, it assumes that every function accepts only one input. Hence, each element in inputs is considered as the only input to every function call. With the value as True, it assumes that every function accepts more than one input. Hence, each element in inputs is unpacked and fed to all functions.
  • input_name (str, optional) – String that sets the index name for the output timings dataframe. This is used later on with plots to automatically assign x-label.
  • indexby (str, optional) – String that sets the index properties for the output timings dataframe. Argument value must be one of - ‘len’, ‘shape’, ‘item’, ‘scalar’.
Returns:

Timings stored in a dataframe-like object with each row for each dataset and each column represents a function call.
Return type:

BenchmarkObj
benchit.extract_modules_from_globals(glb, mode='valid')

Get modules from globals dict.

Parameters:
  • glb (dict) – Dictionary containing the modules.
  • mode (str, optional) – Must be one of - ‘valid’, ‘all’.
Returns:

Extracted modules in a list
Return type:

list
benchit.specs_print(modules=None)

Print system specifications.

Parameters:modules (dict, optional) – Dictionary containing the modules. These are optionally included to setup python modules info and printing it.
Returns:NA.
Return type:None
benchit.specs_short()

Get short-formatted one-line specifications as a string.

Parameters:None – NA
Returns:Specs information as a one-line string.
Return type:str
benchit.setparams(timeout=0.2, rep=5, environ='normal')

Set parameters for benchit.

Parameters:
  • timeout (float or int, optional) – Sets up timeout while looping with timeit that decides when to exit benchmarking for current iteration setup.
  • rep (float or int, optional) – Sets up number of repetitions as needed to select the best timings among them as final runtime number for current iteration setup.
  • environ (str, optional) – String that sets up environment given the current setup with global variable _ENVIRON.
Returns:

NA.
Return type:

None