Understanding Performance Regions

Rico Mariani

·

Follow

8 min read

·

Jun 24, 2018

300

1

Listen

Share

One of the first things I try to teach people about performance planning is that it’s important to understand how your customers will react to various levels of performance and be mindful that your ability to make change in different regimes will be limited and definitely have different costs.

I often ask the question, “Do you need an ‘A’ on this metric or will a ‘C’ do the job?” I think it’s a great question because it leads you to ask “What is an ‘A’ anyway? And what’s a ‘C’ for that matter?”

This is all good thinking.

To help motivate all of this I’ll refer to this hypothetical performance graph which I’m going to discuss presently.

For purposes of this discussion it doesn’t matter what time is being measured here. Actually it doesn’t even need to be time, any resource would do. In fact a resource is probably a better choice than time for analysis, but no matter, let’s keep it simple. The engagement metric can be a measure of any user behavior that means they are “doing the thing.” Again it doesn’t matter what “the thing” is for this discussion, but more of it is better. Let’s dig right in.

Some things to notice

• There is a number, usually not zero, below which there is no data. This is pretty typical and that limit represents the best you can do on the metric in question with the best hardware and circumstances.
• A good looking time distribution often looks vaguely “log-normal” but shifted because of that minimum value. Sometimes I refer to that as the “speed of light” for the scenario.
• There is some point in the PDF where user engagement stops getting better as you make things faster. This generally reflects the reality that at some point it’s fast enough and nobody will notice any further speedup.
• In this particular chart engagement actually starts to go down below a certain value. This is not a mistake, it reflects the fact that in many cases the people that are getting the very best performance are doing so because they are barely using the product. Maybe they tend to visit only toy web pages, or they have only toy text files to edit, or toy solutions to load, or whatever the case may be. The engagement goes down because at least some of those…