Measuring Performance

Pilot draft for author review — September 9, 2026

Who did the better job?

Harbor Service maintains equipment for local businesses. Its technicians travel to customers, diagnose problems, make repairs, and record completed jobs. Imagine you supervise two technicians. During the same four-week period, technician A completes 50 initial jobs and technician B completes 42.

Who performed better?

A completed eight more jobs. If your job as supervisor were simply to identify the larger number, the answer would be easy. But your real task is to decide what those numbers tell you about the business and the people doing its work.

Now you learn that A’s jobs required 10 repeat visits, while B’s required only two. A repeat visit means Harbor sends someone back to work on an initial job. Perhaps the first repair was incomplete. Perhaps an unexpected fault appeared. Either way, returning uses resources that could have been spent elsewhere.

The new information makes A’s lead less convincing. Yet it does not establish that B is the better technician. What if Harbor assigns A the oldest equipment and the most difficult repairs? What if A spends time helping B diagnose unfamiliar problems? What if repeat visits are not recorded consistently?

These are different questions, not reasons to abandon evaluation. We can calculate what the recorded work contributed under stated assumptions. We can also investigate why the results differ. What we cannot do is turn one calculation into an answer to every question about performance.

Harbor, its technicians, and its operating numbers are fictional teaching constructions. Our task is to design an evaluation process for Harbor, not to uncover a fact about an actual company.

The objective, the measure, and the rule

Start by separating three things that are easily confused.

The objective is the result the organization wants. For Harbor, suppose that means providing reliable repairs that customers will pay for, at a cost that leaves the business better off. Completing jobs matters, but so do repair quality, resource use, and the willingness of customers to return.

A performance measure is something the organization can observe and record. Initial jobs completed, repeat visits, and recorded contribution are measures. None is identical to the objective. A job count misses differences in repair quality. A short-period financial measure can miss a damaged customer relationship whose cost appears later.

The evaluation rule determines how the organization interprets and uses the measures. Harbor might review everyone below a job target, compare technicians doing similar work, or ask a supervisor to explain unusual results before assigning a rating. The same job count can support any of these rules. It does not choose the rule for us.

Consider a weekly list of repeat visits. A supervisor could use it to identify equipment that needs better spare parts. Alternatively, Harbor could automatically reduce a technician’s rating whenever a repeat appears. The information collected is similar, but the consequences are different. In the second arrangement, how a visit gets classified now matters directly to the technician.

That distinction matters because evaluation has two jobs. Looking backward, it helps us interpret what happened. Looking forward, it tells people which actions are likely to help or hurt them. A measure that describes yesterday’s work reasonably well may still encourage an undesirable choice tomorrow.

Adding costs changes the ranking

Return to A and B. To make a clear financial comparison, suppose every initial job contributes $100 before repeat-visit costs and incentive pay. This is not the customer’s bill: it is what remains after the initial costs already allowed for in our example. Each repeat visit costs Harbor another $120. Hold other relevant costs equal.

We also need a consistent observation period. Take the jobs completed during the same four weeks and follow each one for 30 days. Count all repeat visits linked to those jobs during their follow-up windows, even when the repeat occurs after the original four weeks. We wait until the final job’s window closes before comparing technicians. Otherwise, recently completed work would have had less time to reveal a problem.

On a small screen, scroll the table sideways to see all columns.

A’s 50 initial jobs produce $5,000 of contribution before repeat costs. Ten repeats cost $1,200, leaving $3,800. B’s 42 initial jobs produce $4,200; two repeats cost $240, leaving $3,960. We do not add repeats to the initial-job counts. They use resources to revisit work already counted.

B leads by $160 on this measure. Another way to see why is to compare the differences. A’s eight additional initial jobs bring in $800, but A’s eight additional repeats cost $960. The extra initial contribution does not cover the extra repeat cost.

This is a better answer to a particular question: what contribution remains from the recorded work after these repeat costs? It is not a complete estimate of the technicians’ value to Harbor. We have not priced mentoring, long-run customer relationships, or failures discovered after the follow-up window. Nor have we established who caused each repeat.

There is an important distinction here between recording a cost and assigning responsibility for it. Harbor incurred the repeat costs whether they arose from poor workmanship or unusually difficult equipment. The financial calculation should not make those costs disappear. But using that calculation to evaluate an individual requires additional reasoning about the work the person faced.

A result is not a direct observation of effort

Suppose Harbor assigns repairs centrally: a dispatcher decides which technician goes to which customer. The dispatcher gives A complicated breakdowns because A has more experience, while B receives routine maintenance jobs. A can make good decisions and work carefully while still completing fewer successful repairs per hour than someone with an easier assignment.

Now change the arrangement. Technicians choose jobs from a shared queue. A worker who wants a high completion count may select short jobs and leave difficult ones for others. Job difficulty still helps explain the count, but it is no longer entirely outside the worker’s choices. Before making an adjustment, we need to know who had the right to choose the jobs.

Keep two other distinctions clear. Outcome noise means conditions and chance affect the result even when the recording is accurate. A replacement part may arrive late despite a timely order. Recording error means the measure fails to capture what actually happened: a completed visit might be omitted or assigned to the wrong technician. Better recordkeeping can fix the second problem without eliminating the first.

An outcome need not be fully under a worker’s control to contain useful information about performance. Additional information can help us interpret it. This is the useful lesson of the informativeness approach to performance evaluation, not a rule that every available statistic belongs in a score. Holmström’s Nobel lecture develops the distinction.

For Harbor, a parts-delay record may explain why a repair remained unfinished. But we would still ask whether the technician ordered promptly, warned the customer, or found a workable substitute. “There was a delay” does not mean “nothing the technician did mattered.” Equally, “the repair was late” does not establish that the technician was careless.

The manager’s task is to ask what information would change the decision. If both technicians had comparable assignments and the difference persisted across several periods, the results would warrant more attention. That still would not turn the contribution calculation into a direct measurement of effort. A longer record helps with some uncertainty; it cannot, by itself, repair consistently different job assignments.

What does the measure encourage people to leave out?

Even perfectly recorded numbers can point effort in the wrong direction. Imagine Harbor counts every initial job accurately and rewards a technician only for increasing that count.

At the end of a visit, a technician can spend a few additional minutes testing whether the equipment works under its usual load. That check might prevent a repeat visit. Or the technician can leave immediately and start another job. The completion count recognizes the second choice sooner than the first. If the technician expects testing to help the customer but not the evaluation, Harbor has created a reason to hurry past it.

Notice that this problem does not require false records or a lazy employee. The technician could work intensely all day while allocating too little time to checking repairs. More effort is not the same thing as better-directed effort.

Economists call this a multitasking problem: when activities compete for time, rewards tied to a well-measured activity can draw attention away from valuable activities that are harder to measure. The central issue is the allocation of attention, not simply whether a person works hard. Holmström and Milgrom’s analysis provides the underlying framework.

Now suppose Harbor adds repeat costs to the score. That gives testing a possible payoff in the evaluation: preventing a repeat protects recorded contribution. But the revised measure creates another question. If technicians can decide whether a return visit is a repeat or a new job, classifying it as new may improve their score without improving the repair.

We can distinguish improving performance from gaming a measure by asking what changed besides the score. Preventing a return visit saves resources. Merely changing the label does not. Choosing easy jobs is more complicated: it may raise genuine short-run contribution, but leave Harbor’s difficult customers unserved. The relevant test is the organization’s objective, not whether the worker technically obeyed the scoring instructions.

It would be a mistake to infer gaming from every favorable classification or every easy job. The point is to identify an opportunity the rule creates, then decide whether the likely consequences justify a safeguard. Suspicion is not evidence.

Today’s target can change tomorrow’s behavior

The way a score is used matters as much as what it counts. Consider two illustrative rules Harbor might adopt. They are alternatives to examine, not existing policies of our fictional firm.

First, suppose a fixed bonus is paid for recording at least 40 initial jobs in a period, with no additional volume payment above 40. A technician at 39 gets a substantial financial benefit from bringing one more job inside the period. Someone already at 41 gets no extra volume pay from recording job 42. The rule has a threshold: crossing one boundary changes the reward abruptly.

If technicians can influence recording dates, the first worker has a reason to accelerate a record and the second may have a reason to save it for the next period. We have not predicted that either will do so. Customer needs, other evaluation criteria, professional standards, and the cost of changing dates may outweigh this incentive. But the volume-pay component treats the same additional job differently depending on where the worker stands.

Now add a second rule: next period’s target equals this period’s reported output. Today’s target is 40. Reporting 40 or 45 earns the same current bonus, but reporting 45 raises tomorrow’s hurdle to 45. If the technician expects that next period’s output could fall between 40 and 44, revealing the higher capacity today can reduce the chance of receiving the next bonus.

This is a target ratchet. Current success changes the standard against which future success will be judged. The resulting trade-off between current performance and future demands is the mechanism analyzed by Weitzman. It can make revealing an improvement personally costly even when the improvement would help the organization.

The lesson is not to freeze targets forever. Suppose a new tool genuinely makes repairs faster. Ignoring that improvement indefinitely could leave a target unnecessarily easy. The manager must distinguish a sustainable change in capacity from a temporary favorable condition, while considering whether the updating rule discourages people from revealing useful information.

For Harbor, one option is to announce that an unusually productive period prompts a discussion rather than an automatic target increase. That sacrifices a simple formula and requires judgment. It also creates room for bargaining and favoritism. We have exchanged one set of problems for another, not found a costless solution.

Three ways to interpret performance

Harbor could compare work against an established standard, against other workers, or through a supervisor’s judgment. These approaches can be combined. Their usefulness depends on which uncertainty matters and what implementing them costs.

Compare against a standard

Absolute evaluation compares performance with a stated benchmark: a contribution target, a completion standard, or a quality requirement. Technicians can see what they are being asked to achieve without waiting to learn how everyone else performed.

The hard part is choosing a sensible standard. If Harbor sets one job target for all technicians, the rule is easy to communicate but may ignore assignment differences. Separate targets for many job categories may be more informative but require reliable classification. They also invite arguments about which category a job belongs in.

An explicit standard is valuable when it makes expectations intelligible. Precision in the wording does not guarantee that the standard represents the right objective or treats different assignments appropriately.

Compare with relevant peers

Relative evaluation compares a person’s result with the results of others. If several crews encounter the same parts shortage, comparing their performance may help distinguish the shared disruption from differences in how they responded. The common-condition rationale is developed by Gibbons and Murphy.

The word relevant matters. A crew handling emergency breakdowns is not automatically a useful benchmark for a crew doing scheduled inspections. Nor do two crews necessarily face the same shortage merely because they work for the same firm. Harbor would need to check the comparison, not simply produce a ranking.

Relative evaluation can also determine who advances. Suppose Harbor has one supervisor opening and awards it to the highest-rated eligible technician. The promotion brings higher future pay and greater authority. This is a promotion tournament: workers compete for a scarce prize based on their relative standing. A technician can improve and still lose if someone else improves more. The rank-order tournament framework comes from Lazear and Rosen.

The prospect of winning can motivate better work today, even if pay in the current job is fixed. It matters whether employees believe additional effort can improve their chances. But consider A spending an hour helping B solve a difficult repair. That helps Harbor while potentially strengthening A’s rival for the opening. A contest can therefore encourage withholding useful help—or undermining a rival—as well as improving one’s own performance. These are possible responses, not claims about what either technician will do. Using peers to understand conditions need not mean making them compete for a single prize.

Promotion also assigns someone to a different job. The strongest repair technician may not be the person best able to coach colleagues, schedule work, and resolve customer disputes. Rewarding current performance and selecting a suitable supervisor are different objectives, as Baker, Jensen, and Murphy emphasize. Chapter 8 returns to that tension and to alternatives such as advancement along a specialist career path.

Use informed judgment

Subjective evaluation gives a reviewer discretion to weigh information that does not fit a predetermined formula. A supervisor might consider a technician’s explanation of an unusual repair, review records of assistance to colleagues, and distinguish avoidable returns from equipment faults.

Judgment can supplement incomplete numerical measures, but employees must have reason to trust how it will be used. A promise to recognize valuable work is less useful if the organization can conveniently disregard it when rewards are due. Baker, Gibbons, and Murphy analyze the role of subjective measures and the credibility of discretionary rewards.

For Harbor, the practical questions are concrete. What evidence must the supervisor examine? Must exceptions have a written reason? Who checks whether favored technicians repeatedly receive generous interpretations? These checks may improve the process, but they take time and can make ordinary decisions cumbersome.

“Objective” does not mean economically complete. “Subjective” does not mean arbitrary. The relevant comparison is between feasible processes, including their mistakes, administrative burdens, and effects on behavior.

Better evaluation is an organizational choice

It is tempting to respond to every weakness by adding another measure. Harbor could record job type, travel time, parts delays, repeat visits, customer comments, assistance to colleagues, and several kinds of repair quality. Each item might be useful in some decision. Collecting all of them for every job could still be a poor use of resources.

Someone must enter the data, check definitions, resolve disputes, and read the results. Technicians who spend more time documenting work have less time to perform it. A supervisor with too many reports may give them less attention. The question is not whether information has value in principle, but whether its expected contribution to better decisions exceeds the cost of obtaining and using it.

This connects measurement to agency costs: costs of dealing with differences between what decision-makers find personally attractive and what the organization wants. Harbor spends resources on monitoring and review, while some losses from misdirected choices remain. A tighter system may reduce one loss while increasing administration or restricting useful discretion. Eliminating every opportunity to make a bad choice is not the same as improving the business.

Measurement also affects organizational structure. If technicians both classify their own return visits and are judged on those classifications, Harbor may assign a separate person to review disputed cases. That changes who has authority, not just which number appears on a report. An independent reviewer introduces handoffs and may know less about the repair. Alternatively, keeping the decision with the technician preserves local knowledge but leaves Harbor needing some way to check it.

The reviewer has incentives too. A supervisor may want favorable-looking team results, avoid difficult conversations, or favor an agreeable employee. Moving a decision upward does not remove the problem of evaluating the person who now makes it. This is why performance measures, decision rights, and oversight belong in the same discussion.

What should Harbor do next?

We can now propose a limited improvement without claiming to have found Harbor’s ideal system.

Begin with the job and repeat records in Table 6.1. Keep the observation window consistent and show both volume and recorded contribution. Do not erase an incurred cost because its cause is disputed, but do not automatically treat that cost as evidence of poor individual performance either.

Before assigning a consequential rating, review the main assignment differences and a sample of repeat visits. Include some ordinary cases, not just the worst-looking records, so the review can detect classification problems as well as explain poor outcomes. Require short reasons for exceptions and give technicians a way to correct factual errors. Sampling saves review time but will miss some problems; broader review may be justified when the stakes are higher.

Then ask what response the revised process encourages. Will technicians test repairs more carefully? Will they avoid difficult jobs? Will supervisors reclassify returns to protect their teams? Those are questions to investigate, not effects we can announce from the arithmetic. Harbor should look for changes in actual service and resource use, not only improvements in the score.

Finally, specify what remains uncertain. B produced $160 more recorded contribution under our assumptions. We do not yet know whether B exerted more effort, whether A received harder work, or whether either technician helped create value outside the recorded jobs. The honest conclusion is narrower than a personnel verdict, but more useful than declaring evaluation impossible.

Chapter 7 takes the next step: deciding how strongly and in what form rewards should respond to evaluated performance. A better measure is an input to that choice. It is not, by itself, a pay policy.

Study map

Start with the organization’s objective. Identify what the measure captures, what it leaves out, and which conditions affect it. Examine the rule that turns the measure into a rating or consequence. Ask how a person can improve the score without improving the business. Compare feasible additions—better records, a benchmark, informed review—against their costs. End with a recommendation that states both the decision it helps and what remains unknown.

Practice and discussion

These are public practice questions, not graded-assignment solutions. Use the chapter’s assumptions unless the question changes them.

1. Change one cost. Suppose a repeat visit costs $80 rather than $120. Recalculate each technician’s recorded contribution. At what repeat-visit cost would the two technicians tie? Explain why changing this cost cannot tell us who worked harder.

2. Investigate the assignment process. You learn that A handles most emergency repairs. Identify two pieces of information that would help interpret the original ranking. Explain how your interpretation could differ if technicians choose jobs rather than receive assignments from a dispatcher.

3. Predict a response. Harbor begins rating technicians partly on the number of customers who submit favorable comments. Describe one way to improve that number by improving service and one way to improve it without a comparable service improvement. Propose a bounded safeguard and identify its cost.

4. Examine a ratchet. Under the chapter’s 40-job target and next-period updating rule, explain why reporting 45 rather than 40 might be unattractive. What changes if extra jobs receive a sufficiently large current reward? Why would a month with only 40 jobs not prove that someone withheld output?

5. Make a recommendation. In about 150 words, recommend a next step to Harbor’s owner before assigning individual performance ratings. Include the objective, one useful measure, one piece of contextual information, one cost of your proposed process, and one conclusion the available evidence does not support.

Check your arithmetic

For question 1, A contributes $5,000 − $800 = $4,200 and B contributes $4,200 − $160 = $4,040. A now leads by $160. A’s eight extra initial jobs bring $800 more initial contribution, so the rankings tie when the eight extra repeat visits cost $800: $100 per visit. At that cost both contribute $4,000. These are changes in the recorded-contribution comparison, not discoveries about effort.

Source note

The linked readings ground the short theoretical propositions; the Harbor narrative, numbers, proposed procedures, and exercises were constructed for this book. They are not reported findings from those papers.