Evaluations of "Evals"

Nick and Nate are spending time evaluating our current Design Studio evaluation system over the next few days. Last night's post struck a chord with me. It was pretty similar to discussions I had with Ron Jeffries and relates to things I talked about in my Human Resources class. This post is to expand my reflection's on Nick's post.

An evaluation comprised of interviews could potentially put more pressure on the evaluator. However, I think this could be a good thing. In any interview situation, being able to express ideas clearly, sense the emotional and rational state of the interviewee, and maintain an ebb and flow of control in the interview as necessary. Frankly, I can't think of a reason a PM shouldn't develop skills like these.

As far as being difficult to understand, the current system is the same way. Whether spoken or written down, team members are still offering insights to the performance of other team members and it is the job of the PM to understand, compile, and analyze the evaluations of the team members. With a notepad, a good pen, and perhaps a tape recorder, an interview style evaluation would reveal the same--if not more--information as the current system.

Furthermore, I don't see understand why it is the PM's responsibility to defend anybody. Being a rational arbiter in a conversation is an important point, and it is important to have a clear-headed evaluator in the case where emotions are running high.

Moving on, I think interview style evaluations with shorter iterations would be more effective. If evaluations should be focused on how a student can improve, there should be more than four opportunities to get any real feedback or direction. This makes potential for drastic or severe changes if a student really wants to respond to feedback. If the iteration periods were shorter, a student could focus on one or two points of improvement, adjust, and move on until the next evaluation. With what we have now, any high points or low points are moderated over time to the point where whatever is being awarded or admonished has been forgotten (yay for passive voice). One of the important points impressed upon students in psychology or organizational behavioral management is that timeliness is hugely important when rewarding or correcting behavior. Another advantage of shorter iterations is that it reduces the potential that students use evaluations as a shotgun trigger to hurt other team members. There is certainly the possibility that a student who wanted to avoid confrontation would suppress any problems or issues until evals roll around and then they could blast away as they wanted. Evals should be about improvement, not venting. Therefore, in a shortened evaluation period, time no longer becomes ammunition and the evaluations offer more value to a student who truly seeks to improve.

Interviews can reveal more information than current evaluations. How many students are actually honest or specific in the current evaluation system? Suppose an evaluation reveals you did a good job in a certain area, but can't offer any information on what specific activities or instances contributed to that evaluation. Or on the contrary, suppose a student gets a negative review, but has no idea why or for what specific reasons. In an interview, a PM is in a good position to ask questions or elicit more details to get a full picture.

One drawback of interviews with shorter periods is that it means more work for the PM. Having to arrange interviews, conduct them, compile the information from each, and present reports back to students would be a lot of work in an interview. However, I think the benefits may outweigh the work here. PM's should certainly be on board with giving their team members better opportunities to respond, grow, and improve.

Another issue is that interviews may seem difficult to quantify or provide any gradable substance. First and foremost, evaluations should be about progress, not grading. DS is in a unique position in that we have to assign grades in an environment not normally driven by academic growth. In fact, I can't think of any actual growth I've experienced from Design Studio as a class.

Therefore, it would be important to identify clear goals and objectives for a Design Studio student. In the same way that a class syllabus or an overview for a chapter in a text books describes what the student should have achieved by the end, a set of Design Studio goals and objectives for a student should be expressily outlined. It should also be something reiterated and impressed into the mind of the student. If Design Studio currently has goals like I'm talking about, then we're not stressing them because I have no idea what they are off the top of my head. In what areas am I expected to grow because of this class? What are the measurable points of success in this class? What skills should I have because of my two years in Design Studio and all the hours in team time?

The answers to these questions should be the same questions an interviewer should seek to discover in an interview. This would give the PM some structure to the interview. Facilitating a two-way discussion shouldn't be difficult as the PM asks a student how another team member is doing on a certain point of success. Really, we do that now in written form, so that isn't such a big leap.

Things like communication, leadership, diplomacy, handling tough situations, persuasiveness, and other good traits for the workplace are things we seek for now and are things we should continue. Identifying core values for a Design Studio student would be a key step. Putting more emphasis on goals would be easier in an interview since I don't think anybody pays much attention to those now.

Another issue with the current system is there close to no value of meeting with the PM after evals now. The PM usually sends the student a copy of the results of their evaluation about an hour or so (if that) before a meeting. Then the PM reads the review to you. There is no value in that. I can read. I would see more value out of my and my PM's time if we were actually discussing and poring over questions when the questions mattered to me most--ie, when things happen or shortly thereafter.

Well, those are my thoughts. Either way, I'm gone after this year, so no big deal :)