Code Review Dashboard

The Xen project has developed a code review Dashboard in collaboration with Bitergia. There are two variants of this dashboard

• kibana.bitergia.com/xen/app/kibana: User version
• kibana.bitergia.com/xen-edit/app/kibana: Administrator version (allows to create extra customizations]

This document provides a version of the documentation for the*user version* of the dashboard, that is editable and easily consumable

Panels

The Xen Project Code Review dashboard has 4 panels: One for documentation, a Community Panel, a Performance Panel and a Backlog Panel.

Community Panel

In this panel, several widgets are grouped by use-case.

Use case: Identify top reviewers (for both individuals and companies)

Evolution 'Reviewed-by' flag (no patchseries, no patches)

Shows the evolution of the Reviewed-by flag over time. Allows to create a time filter.

Top People and Domain Reviewing Patches

Counts the number of people and organisations who commented on other people’s patches (by counting the Reviewed-by attributes) in the selected time period. Allows to filter by developer (e-mail address and e-mail domain).

Use case: Identify Imbalances between reviewers and contributors

Evolution Patches (no patchseries, no comments)

Shows the evolution of the patches posted over time. Allows to create a time filter.

Top People and Domain Imbalance Analysis

Over the selected time period, the following fields are aggregated per person or domain * Balance: Difference between patches posted and and comment activity. A positive number implies that the person or organisation comments more than she posts patches. * Patches: Number of patches posted * Reviews: Number of Reviewed By flags

Use case: Identify post-ack comments on patches

Evolution Comments Activity (no patchseries, no patches)

Shows the evolution of comments made on patches (both real and self-comments) over time. Allows to create a time filter.

Top People and Domain Commenting on Patches

Over the selected time period, the following fields are aggregated per person or domain * Comments: The number of review comments on other people’s patches. * Post ACK comments: The number of comments made after a patch has been ACK’ed

Use case: Drill into Review activity information

The following widgets are of particular interest to code submitters, reviewers and maintainers. A similar view, which is optimised to identifying which code reviews are almost complete are visible in the Backlog and Data-Explorer panels.

Evolution Email Activity (patchseries, patches, comments)

Shows the evolution of review related e-mail activity over time. Allows to create a time filter.

Evolution Patch Series (no patches, no comments)

Shows the evolution of patch series posted over time. Allows to create a time filter.

Patch Series

Lists the patch series in an exportable table that satisfy selected filters.

Reviews (using Reviewed-by flag)

Lists emails that set the Reviewed-by flag that satisfy selected filters.

Comments

Lists emails comments that satisfy selected filters.

Performance Panel

The data presented in this panel allows us investigate merge time, complexity of patches, number of review comments and other related metrics which indicate complexity and correlate the different properties.

Important note: this panel only shows patch series that were merged, as many of the metrics in this panel only apply to merged series.

Each section below, gives a brief overview of the widgets that are present in this panel.

PatchSeries

Counts the number of patch series

PatchSeries per Year

The pie chart represents the total number of patch series that were merged per year. When clicking on some of the portions of the pie, this will create a time based filter for that specific year.

Evolution NumPatches per PatchSeries

Total number of patches for selected patch series over time, arranged in percentiles. An indirect measure for complexity of patch series submitted over time.

Several charts (including this one), show data which are divided into 3 percentiles. The datasets are ordered by a specific metric, (in this case the total number of patches merged) and are grouped up to the 25%, the 50%, the 75% percentile and in some cases the 95% percentile.

This approach is easiest to explain by example: if your 3-month-old baby is in the 25th percentile for weight, that means 25 percent of 3-month-old babies weigh the same as or less than your baby, and 75 percent weigh more. The higher the percentile number, the bigger your baby is compared to other babies the same age.

Ability to create a time based filter.

Time to merge per number of patches per patch series

A histogram that shows the time to merge on the y axis and the number of patches per series on the x-axis. The histogram is sorted in order of descending population (like a Pareto chart), with values that occur most often on the left and values that occur least often on the right.

Creates a filter for the num_patches = <selected column on y axis>

Evolution NumVersions per PatchSeries

Total number of re-posted versions for selected patch series over time, arranged in percentiles. An indirect measure for the degree of actionable feedback and possible quality/complexity issues on the patch series submitted over time.

Ability to create a time based filter.

Time to merge per versions of patches per patch series

A histogram that shows the time to merge on the y axis and the number of reposts per series on the x-axis. The histogram is sorted in order of descending population (like a Pareto chart), with values that occur most often on the left and values that occur least often on the right.

Creates a filter for the num_versions = <selected column on y axis>

Evolution NumVersions per PatchSeries

Total number of re-posted versions for selected patch series over time, arranged in percentiles. An indirect measure for the degree of actionable feedback and poss

Backlog Panel

Evolution Merged(1) and Non Merged (0) PatchSeries

Shows the evolution over time of merged and un-merged patch series. Note that we cannot currently identify patch series to other projects (e.g. Linux, QEMU, …) that are cross-posted on xen-devel@ as merged, even if they have in fact been merged.

Top People and Domains that got merged

Lists the number of people and organisations that were merged.

Patch Series Merged and Not Merged (Backlog)

Lists the patch series that were merged and not merged

Use case: Identify recently active patch series

Use one of the following pre-defined time filters to identify review activity in a certain time period (as defined below):

Hot      = [now to 1st week}
Warm     = [1st week to 1st month}
Tepid    = [1st month to 2nd month}
Cold     = [2nd month to 5th month} 
Freezing = [5th month to 1 year}
Dead     = [1 year to *} 

Check the Patch Not Merged (Backlog) widget to identify series that were not merged.

Use case: Identify abandoned patch series (those older than 1 year)

Use the Dead time filter and check the Patch Not Merged (Backlog) widget to identify series that were not merged.

Use case: Identify patch series in progress

Check the Patch Not Merged (Backlog) widget or merged:0 filter to identify series that were not merged.

Use case: See whether the backlog is increasing

Use the Evolution Merged(1) and Non Merged (0) PatchSeries widget together with the merged:0 and merged:1 filter to identify trends

Use case: Focus attention on nearly completed reviews

Use the following filters to identify patches that are nearly complete. Note that these filters can also be used in the Data-Explorer Panel: * patchserie_percentage_ackedpatches (number): contains the % of acked patches in a specific patch series. Numbers may range from 0 to 100. Use the [90 TO 100} notation for filters. * patchserie_numackedpatches (number): contains the number of acked patches for a particular patch series * patchserie_numpatches (number): contains the number of patches that make up a particular patch series

Data-Explorer Panel

This view is tailored towards code submitters, reviewers, maintainers, committers and release managers. Documentation is available in the panel itself. It can also be used as a more detailed view into review related messages using the techniques outlined in the Backlog panel.

Index-Review and Index-Time Panel

This view provides raw access to the raw xen-patchseries-reviewers and xen-patchseries-timefocused indexes and can be used to explore the raw data behind other panels.

Basic Operations

Filtering

Setting the Time Filter

The Time Filter restricts the search results and all widgets within a panel to a specific time period. By default the time filter is set to the last 5 years. You can use the Time Picker in the upper right corner of the menu bar to open the Time Picker. Click the caret at the bottom of the Time Picker to hide it.

Quick Filter: To set a quick filter, simply click one of the shortcut links. The quick filter contains a number of pre-defined time periods. We added a number of Xen Project specific quick filters: * Years, to make selection of activity in a specific year easier * A set of time periods (Hot, Warm, Tepid, Cold, Freezing and Dead) which are relevant to identify review activity in a certain time period (as defined below):

Hot      = [now to 1st week}
Warm     = [1st week to 1st month}
Tepid    = [1st month to 2nd month}
Cold     = [2nd month to 5th month} 
Freezing = [5th month to 1 year}
Dead     = [1 year to *} 

• A set of time periods, which match Xen Project release cycles (Important Note: These Xen Project specific quick filters are not yet implemented).

Relative Filter: To specify a relative Time Filter, click Relative and enter the relative start time. You can specify the relative start time as any number of seconds, minutes, hours, days, months, or years ago.

Absolute Filter: To specify an absolute Time Filter, click Absolute and enter the start date in the From field and the end date in the To field.

You can also set an absolute time filter by selecting a time interval in any widget that has time in the x axis (they typically are named Evolution of XYZ) . Click and drag to select a specific timespan. You must start the selection with the cursor over the background of the chart: the cursor changes to a plus sign when you hover over a valid start point. You can use the browser Back button to undo your changes.

Working with Filters

Widgets

Some widgets (check the widget descriptions in Help: Panels) allow you to create a filter. When you create a filter anywhere in Kibana, the filter is displayed in a green oval under the search text entry box. Hovering on the filter oval displays icons to enable, pin, toggle, remove and customise the filter. For more information see Discover Filters.

Search Widgets

In search widgets (their name always starts with Search), you can use any field in any data record (also called document) to create a filter. For more information, see Filtering by Field.

Advanced Filters

Search field and query syntax

The search field underneath the menu bar provides a way to query a specific subset of transactions from the selected time frame and set filters. The query syntax is based on the Lucene query syntax. Also see Kibana Queries and Filters for additional background info.

Important notes:

• To reset a search, add * into the search field.
• Sometimes, complex search queries fail, unless the search field has been reset
• The type of field (string, number, date, etc.) has an impact on the query syntax
• Queries are case sensitive
• To construct sensible queries, please check the information in Help:Indexes and fields. You can only create queries that use fields within the respective index that is used in a panel: if you use a field that does not exist, the search will return No results found.
• Fields that are not available in an index are defaulted to -1 for numbers and na for strings. The only case where this is not true, is the balance field in the xen-patchseries-reviewers index: this field is defaulted to 0, when not calculated.

Advanced Filter Examples

Below are a few examples of queries

patchserie_id: [6000 TO 6100]

num_patches: 1

patchserie_id: [6000 TO 6100] AND num_patches: 1

Note that expressions such as num_patches:> 2 (aka >, >=, <, <=) do not always work reliably. If they don’t, use range searches (e.g. num_patches: {2 TO *]) instead.

patchserie_id: [6000 TO 6100] AND num_patches: {2 TO *]

patchserie_percentage_ackedpatches: [50 TO 100]

patchserie_numackedpatches: [1 TO *] AND patchserie_numpatches: [3 TO *]

patchserie_numackedpatches: [7 TO *] AND patchserie_numpatches: [10 TO *]

Additional Resources

Definitions

There are two indexes, whose fields can be used in advanced filter queries. The indexes are * xen-patchseries-timefocused: The Performance panel is based on this index. The Index-Time panel exposes the raw data behind this index. * xen-patchseries-reviewers: The Backlog, Community and Data-Explorer panels are based on this index. The Index-Review panel exposes the raw data behind this index.

Tables

Data in tables can be exported as a comma-separated-values (CSV) file. To do this, click on either the Raw or Formatted links at the bottom of any of the detailed information tabs. A raw export contains the data as it is stored in Elasticsearch. A formatted export contains the results of any applicable Kibana field formatters.

Mapping data to e-mails

Both indexes contain a message_id field, which * can be used to retrieve the message in an index in your inbox (please refer to your mail client’s documentation) * can be used to find a message via marc.info. Note that in most series, patch, comment, etc. related tables, there are direct links to marc.info e-mails

We use the following syntax is to map the message_id field to e-mails:

http://marc.info/?i=<message_id>

If the message_id=<1455681279-28451-1-git-send-email-jfehlig@suse.com>, then the relevant search query is

http://marc.info/?i=<1455681279-28451-1-git-send-email-jfehlig@suse.com>

Additional Resources

The following articles provide useful background reading on using Kibana based panels:

• Dashboard describes all functionality that is available in Dashboards or Panels. Note that any functionality that allows changing and saving dashboards is disabled in kibana-xen.bitergia.com. If you want specific customisations, please contact community.manager@xenproject.org.