A Consultant's Casebook

              about me:   Andres Inn, Ph.D.

Information Leakage

Every serious candidate for promotion will obtain as much information as possible regarding the Assessment Center that could determine his salary, his benefits, and perhaps his future with the organization.  And every serious candidate will have some friends and colleagues who could advise him regarding the content of an Assessment Center.  And when Assessment Centers are held (as usual) over a period of many days, it is more likely that later candidates receive more information than earlier candidates.  

Such information can have both direct and subtle effects.  If I know that racism is an issue that will be judged, I can exhibit "proper" behaviors and attitudes to the raters.  If I know that there is an in-basket" exercise in which candidates must respond in writing to the messages placed in their "in-basket," I can anticipate how these behaviors will be judged:  I should prioritize messages, I should understand the chronological order of events, I should delegate responsibility where appropriate and make decisions when needed.  The moment I know there is a "leaderless group discussion" exercise, I know I need to stand out as a leader.  I must not dominate the discussion, but should be seen as guiding the discussion, and so on.  If I know the topic under discussion, I can even prepare a chronology of issues to discuss in a logical sequence.

But we should not discount the subtle effects of information.  If I believe that I am better prepared with information than my fellow colleagues, I will inevitably display more confidence and self-assurance.  These are leadership qualities that raters are sure to recognize and reward with points.  

The effects of information leakage can be dramatic.  In the case of John Teahan's assessment procedures with the Detroit Police, we saw this problem manifested as score inflation over the 6 days of Assessments.    

The argument for use:   The assessment task was made more difficult because the assessors could not get adequate job descriptions, the agents may be sent to do tasks different from the ones originally intended, and the shifting theaters of war may lead to a complete change in the situation to which the recruit was trained for and sent.  Same problem with managers.  Can't get adequate descriptions for how managers "lead," "motivate," "organize," etc.  Worse, different managers may use completely different technologies for accomplishing the same results.  Assessment Centers are supposedly able to assess the flexible approach assumed to be necessary for management. 

Go on to rationale for assessment centers.  Choose actual tasks hat require display of actual on-the-job stuff.  Police Oral Boards --> voila transition done.

Where does the if he can do this, he can do that fit in?  If he can do it once, he can do it twice.  Been here, done that, can do it again.  

Many kinds of data are serially correlated.  If the temperature today is -10, it is extremely unlikely that tomorrow will register +30.  In the past few years, I've amazed some friends by predicting winter weather during the summer.  My predictions have been easy because the summers were unambiguous predictors.  A few years ago we experienced an incredibly long, dry summer; so I predicted a short warm winter.  Then we suffered a short, cold, and wet summer after which I predicted a long, cold, and snowy winter.   I couldn't go wrong!

But the point is that human performance is likewise serially correlated.  To know how successfully someone will complete a future assignment, look at his past assignments.  That's why successful managers and CEOs command so much money when they move from one company to the next.  Their future performance is expected to be at least as good as their past performance, and it's a relatively safe bet.  The assessment centers purport to capitalize on this fact and attempt to present "face validity."  That is, the situational exercises appear to have some relationship to managerial tasks, and assessors assume that candidates who perform well on assessment center exercises will perform equally well on the job and in the future.  This argument rests on two assumptions:  (1) that assessment center exercises are truly comparable to job requirements, and (2) that assessors accurately measure performance and not some irrelevant variables.

It's the time span!  1-2 days of assessment center dubiously transferable tasks vs years of measurable experience.  measurable .... see police performance appraisal.

Then there's the problem of examples.  The best predictor of successful job performance is previous job performance.  Anecdotal evidence from the early German, British, American assessment procedures suggest that those who passed through the assessment procedures also performed successfully.  Not only that, but those who passed, were able to excel in a broad range of other activities.  Recall "Wild Bill" Donavan, Wall Street lawyer - war hero turned spy master.  Or Julia Child, a famous graduate of the British Camp X, who mastered cooking, book publishing, and television presentations.

But recall also, that these people were already successful before the assessment procedures began.  First, they were all of exceptional intelligence.  ... so what is it that assessment centers measure?  Is it merely intelligence.  Beyond our scope to .... but it's an interesting thought.

Face Validity

So, how well do the situational exercises mirror the requirements of actual jobs?  Decide for yourself how well the Brook Exercise, the Wall Exercise, the Construction Exercise, and the Stress Interview mirrored the actual requirements of spies, saboteurs, and field agents in Nazi Germany.  As regards the standard fare of assessment centers for managerial candidates, I remain unconvinced.

First, there is the problem of duration.  An assessment center exercise which takes an hour or two cannot compare to the tasks of managers which require a focus on results for a period of months, quarters, or even years.  Short-term focus on an assignment with a concrete end has nothing to do with operating in a business environment where the shifting sands of competition, personnel, profits, exchange rates, politics demand a long-term focus on performance goals.  

Second, there is the problem of rater focus.  Assessment Center raters measure simple observables:  whether or not the candidate prioritizes, whether or not the candidate promotes her ideas forcefully, whether the candidate organizes and 

The In-Basket Exercise  

The Leaderless Group Discussion

Superior/Subordinate Interactions

Actually managers don't sit at their in-basket all day and prioritize memos and decide on telephone calls to make.  And managers don't organize "leaderless" group discussions, rather they organize decision making groups which they lead!  And, I believe that the primary qualities of executives are completely missed by the Assessment Center exercises.  These regard long-term goal directedness, and memory -  memory for the details of social interactions in the office.  Managers and real organization leaders remember who has promised what and when.  They remember who has performed well in the past and who can be relied on to perform well again.  And they do this in a social context - over lunch, casual meetings, during coffee breaks.  They use social persuasive skills to organize the work collective toward the long-term goals that the leader/manager has identified.  It is the skill in using the social context and the peer-to-peer, superior-to-subordinate, and subordinate-to-superior relationships at work to steer the organization toward goals.

These are not necessarily intellectual skills.  During my lifetime I can think of only a few really outstanding leaders - men and women who moved organizations in a new direction against the incredible inertia exercised by their bureaucracies.  In the political realm I regard Margaret Thatcher, Helmut Kohl, and Ayatollah Khomeini as truly great leaders.  Each of these leaders was able to move their governments and people in a new direction (whether or not they wanted to go in that direction).  Margaret Thatcher dismantled much of the state-owned corporations in the UK.  Helmut Kohl almost single-handedly decided that East and West Germany should be re-united and committed untold amounts of West German marks to that end.  Ayatollah Khomeini imposed an in-ward looking theocratic government on relatively well-educated and outward-looking Iranians. 

Likewise, who dares argue with me that a rater who observes my behavior in the contrived setting will be able to evaluate my performance better than the people who have worked with me over the years?  

All material is copyrighted!  Not to be used without the written permission of Andres Inn.  ©2000.

Send mail to Andres.Inn@mail.ee with questions or comments about this web site.
Last modified: October 19, 2000