Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
124 changes: 118 additions & 6 deletions introducing-metrics-in-innersource.md
Original file line number Diff line number Diff line change
@@ -1,16 +1,128 @@
**Note: This pattern is currently under development in [this Pull
Request](https://github.com/paypal/InnerSourcePatterns/pull/83)**
# Title

## Title
Introducing Metrics in InnerSource

Introducing Metrics in the InnerSource Initiative

## Patlet
# Patlet

Involve all stakeholders in designing and interpreting metrics to measure the
current status in terms of _health_ and performance of the InnerSource
initiative.

# Context

An organization is planning to apply or this is in the early stages of applying InnerSource. This would like to measure the current status in terms of 'health' and performance of the initiative, and if the expected outcomes such as an increase in the level of cross-divisional and cross-location collaboration are actually taking place.
This pattern applies to early stages of the initiative or are small in their scope, but they may be mature in their initial process and steps.


# Problem

This organization may already have qualitative feedback from the
involved teams, but desire more objective information focused
on development activities.

The organization does not really know where to start measuring
things or what are the key parameters to measure.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it make sense to state the goal of the measurements in the problem statement? Perhaps something along the lines of The organisation would like to measure the health and performance of the InnerSource initiative but does not really know [...]. Or maybe The organisation would like to measure how the InnerSource initiative increases the level of cross-divisional and cross-location collaboration, but does not really know [...].

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just added this to the text. Is that how you expected to be?

Changes in the top level initiatives may affect the InnerSource program
as they rely in the good will of some executive from the organization.

You may have a problem justifying the InnerSource effort when there is
a change in business priorities or business leadership. Then you need
something concrete to justify the program. A future problem you're
guarding against.

If there's a change in the C-level, metrics might be helpful to convince
them that InnerSource is useful.


# Forces

People do not like to be tracked or measured.

There is not a proper monitoring infrastructure for the software
development process and this is hard to build or to get funding
for this.

There is not a culture of software development metrics.

Metrics are usually misunderstood if people have not received any
training on those.

Copy link
Copy Markdown

@gregswindle gregswindle Nov 23, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🙋‍♂️ Suggested addition

Organizations have unrealistic (or premature) business objectives. (Needs an example.)

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggestion added to the text. Does it work for your? Thanks for the suggestion.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks!

Organizations collect vanity of any other type of metrics that do not
track business objectives' sucess or failure over time.

Metrics tend to become goals, will subsequently be gamed and thus meaningless.

Merging existing team metrics if another team has a way to measure how teams are doing; these new measurements could potentially conflict.

Some countries in some organizations may face extra complexity when introducing metrics as they may not allow to track individuals.

Tools and how they use them. There might be a learning curve in the discussion about metrics. And perhaps the tools do not support the metrics we're looking for.

# Solution

Bring developers, middle managers and C-level to have a discussion
about metrics. And consider other roles out of the usual development process such as
Human Resources, legal departments, product management, and others.

Let developers and middle managers know that these metrics or KPIs
are not focused on tracking their personal performance, but to compare
if the initiative is currently working as expected.

Consider third party and neutral player to produce such metrics.

Have specific training on the topic of metrics and good practices
to use them. An example is to have a methodology to follow metrics such
as the Goal-Question-Metric approach or the Objetives-KeyResults one.
On the other hand, try to update the metrics used to the short-term
and medium-term goals.

Bring specific discussions on the metrics to be used avoiding per
developer granularity or at least anonymizing that info.

Produce a characterization of metrics as this might be helpful for others
to understand and follow.

* Nearly always InnerSource is not a goal in-and-of itself but a proposal of how to improve some larger problem that the company is having. One class of metrics is around that larger goal (e.g. quality, interrupt-driven work, duplicated code, etc.) and how it is going for the company.
* Another class of metrics is around how much InnerSourcing is happening. A raw definition of InnerSource based on code submission to a repository not owned by the submitter's team could be measurable in a few ways.
* A third class of metrics are KPIs that we believe will improve the raw amount of InnerSourcing happening (e.g. mean-time-to-review, automated test run on PR, etc.).

Note: any proposed metric are just examples and not the ones that should be using. Depending on the selection of business goals to track, those will match with specific set of metrics.


# Resulting Context

The organization builds a monitoring infrastructure where the agreed
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

agreed metrics

If part of the solution is that all (?) stakeholder, including developers, need to agree on metrics, that should probably go in the solution section, too. Seems important.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@gruetter, I really like the recommendations in "When, What and How to Measure" (as you suggested, @dicortazar).

Great work, BTW, @dicortazar!

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @gregswindle !! Does it make sense to elaborate a bit more on the method and strategy? Or does it make sense if we open a new pattern to deal with that specific topic and then link that pattern from here? (I like this second idea... but not sure how to proceed).

metrics will be the ones to be used.

Those metrics will help C-level to understand the current situation
of the project and this will help to compare the status before
applying inner source and after applying the inner source initiative.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wouldn't that require baselining? If yes, should that go in the solution section, as well?

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What does everyone think about applying the same metrics to "traditional" internal software development, and using "traditional" development for baselines?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wouldn't say baselining and that sounds more like historical metrics, but at least have metrics prior and after an event took place.


Those metrics will help middle management to understand how the
initiative evolves and help them with their daily work.

Those metrics will help developers to understand how the initiative
evolves and help them with their daily work.


# Known Instances


# Authors

- Daniel Izquierdo
- Tim Yao
- Clint
- Russ Rutledge
- Tom

# Acknowledgement

- Georg
- Bob

# State

Early Idea