3. Performance diagram

3.1. Description

Performance diagrams are used to show the relationship between categorical statistics, with axes representing detection and success(1 - false alarm) rates Roebber, 2009). The simplest input to the performance diagram is the MET contingency table statistics (CTS) output. This output can be produced by many of the MET tools (Point-Stat, Grid-Stat, etc.).

There are several reference lines on the performance diagram. The dashed lines that radiate from the origin are lines of equal frequency bias. Labels for the frequency bias amount are at the end of each line. The diagonal represents a perfect frequency bias score of 1. Curves of equal Critical Success Index (CSI) connect the top of the plot to the right side. CSI amounts are listed to the right side of the plot, with better values falling closer to the top.

3.2. Example

Sample Data

The sample data used to create these plots is available in the METplotpy repository, where the performance diagram scripts are located:


$METPLOTPY_SOURCE is the directory where the METplotpy code is saved. The data is text output from MET in columnar format.

Configuration Files

The performance diagram utilizes YAML configuration files to indicate where input data is located and to set plot attributes. These plot attributes correspond to values that can be set via the METviewer tool. YAML is a recursive acroynym for “YAML Ain’t Markup Language” and according to yaml.org, it is a “human-readable data-serialization language. It is commonly used for configuration files and in applications where data is being stored or transmitted”. Two configuration files are required, the first is a default configuration file, performance_diagram_defaults.yaml that is found in the $METPLOTPY_SOURCE/METplotpy/metplotpy/plots/config directory. $METPLOTPY_SOURCE indicates the directory where the METplotpy source code has been saved. The second required YAML configuration file is a user-supplied “custom” configuration file that can be used to customize/override the default settings in the performance_diagram_defaults.yaml file. The custom configuration file can be an empty file if all default settings are to be applied.

Default Configuration File Settings

  • title The title for the performance diagram

  • title_weight 1=plain 2=bold 3=italic 4=bold italic

  • title_align Unsupported by Matplotlib, included for consistency with METviewer

  • title_offset Unsupported by Matplotlib, included for consistency with METviewer

  • title_size Magnifier value. Values above 1.0 create title that is larger than the internal default size. Values less than 1.0 create a title that is smaller than the internal default size.

  • xaxis The label to the x-axis

  • yaxis_1 The label to the bottom y-axis

  • yaxis_2 The label to the top y-axis (leave empty if second y-axis is not needed/required)

  • plot_width width of plot in inches

  • plot_height height of plot in inches

  • plot_units units for plot: in for inches or cm for centimeters

  • plot_ci A list of values of the type of Confidence Intervals to apply. Choose either None for no confidence intervals or Norm for normalized confidence intervals.

  • plot_disp A list of True/False values. True to display, False otherwise

  • series_order A list of values 1..n indicating the order of the series in the list e.g. if the list is 3, then 1, then 2 this indicates that the first series in the list is to be treated as the third, the second element in the list is to be treated as the first series, and the third element in the list is to be treated as the second series.

  • indy_var The independant variable

  • indy_vals A list of independent variables

  • fcst_var_val_1 Variables of interest

  • fcst_var_val_2 Second set of variables of interest

  • series_val_1 Series values

  • series_val_2 Second set of series values. Leave as empty

  • list_stat_1 List of statistics of interest

  • list_stat_2 Second list of statistics of interest. Leave as empty list

  • user_legend List of legend labels. One for each series. If any list member is empty, a legend label will be created based on other information.

  • legend_box ‘n’ for no box around the legend ‘o’ for a box drawn around the legend

  • legend_ncol Integer value indicating how many columns of legend labels

  • legend_inset x and y values indicating position of the legend

  • legend_size A magnification value. Value greater than 1. Produces a legend that is greater than an internal default value. Value less than 1.0 produces a legend smaller than an internal default size.

  • plot_stat The statistics to plot: median, mean or sum

  • plot_contour_legend True for drawing a legend for contour lines, False otherwise

  • colors A list of colors one for each series. Define as color name or hexadecimal values

  • series_line_width A list of widths for each series line. Values greater than 1 result in a thicker line.

  • series_symbols A list of symbols to apply for each series: ‘o’ for circle, ‘s’ for square, ‘H’ for ring, ‘d’ for diamond, ‘^’ for triangle

  • series_line_style A list of line styles to apply to the corresponding series: ‘-’ for solid line ‘–’ for dashed line ‘:’ for dotted line

  • event_equal True to perform event equalization on data, False otherwise

  • annotation_template Annotation for y-value. Leave empty if no annotation is desired. Otherwise indicate template with “%y <units>”. Double quotes around annotation is needed.

  • plot_caption Caption text, leave empty if no caption is desired

  • caption_weight 1=plain text 2=bold 3=italic 4=bold italic

  • caption_col Color of caption, color name or hexadecimal value

  • caption_size Relative magnification of an internal default font size

  • caption_offset The up/down position relative to the x-axis

  • caption_align The left/right position relative to the y-axis

  • xlab_size Size of the x label as a magnification of an internal default size

  • xlab_align up/down positioning relative to x-axis

  • xlab_offset left/right position relative to the y-axis

  • xlab_weight 1=plain text 2=bold 3=italic 4=bold italic

  • xtlab_orient unsupported by Matplotlib, kept for consistency with METviewer

  • xtlab_size unsupported by Matplotlib, kept for consistency with METviewer

  • ylab_align left/right position of y label

  • ylab_offset up/down position of y label

  • ylab_weight 1=plain text 2=bold 3=italic 4=bold italic

  • ytlab_orient y-tick label orientation

  • ytlab_size size of y-tick labels as a magnification of an internal default size

  • stat_input path and filename of the input MET stat file

  • plot_filename path and filename of the output performance diagram PNG file Only PNG output is currently supported.

3.3. Run from the Command Line

To generate a default performance diagram (i.e. using settings in the performance_diagram_defaults.yaml configuration file), clone the code from the METplotpy repository at github.com/dtcenter/METplotpy:


git clone https://github.com/dtcenter/METplotpy

Change directory to $METPLOTPY_SOURCE/METplotpy/metplotpy/plots/performance_diagram, where $METPLOTPY_SOURCE is the directory where you cloned the code.

cd METplotpy/metplotpy/plots/performance_diagram

Activate your conda environment, which has all the Python requirements outlined in the Installation guide.

Run the following at the command line:

python performance_diagram.py ./minimal_performance_diagram.yaml

This will create a PNG file performance_diagram_default.png in the directory from where you ran the python command above.

To generate a slightly modified plot, re-run the above command using the custom_performance_diagram.yaml file:

python performance_diagram.py ./custom_performance_diagram.yaml

This will create a PNG file, performance_diagram_custom.png, which will differ in appearance from the default plot. These plots use the plot_20200317_151252.data that is found in the METplotpy/metplotpy/plots/performance_diagram directory, and creates the PNG plot in addition to a plot_2020-317_151252.points1 file. The latter is a text file that contains the x- and y-values that are being plotted and is useful in debugging.