THE JUDGING PROCESS – PART 2 – PLACING THE DOGS AND PICKING THE WINNER

Dennis Voigt interviews Ted Shih and Terry Rotschafer

to find out how they determine winners after their Callback Process that they detailed in the last issue.

 

Initial Determinations

Dennis: In the last issue, you both gave us an excellent synopsis of the process that you use to quickly, efficiently and fairly do callbacks during a field trial. There was food for thought there for all judges. An equally big challenge as the callback process is the task of determining your placements and getting the right winner. There are many trials where the placings are not obvious, at least to the gallery. I think it would be valuable if I asked you about different ways of doing this including how you tackle some of the more difficult decisions.

            My first question is how you start. Personally, I prefer to sit down first and go through my finalists alone without my co-judge. Then, when I have my initial decisions as best I can without discussion, I like to start working on agreement. I especially want to have picked my winner at this stage. Others do it differently. How do you start initially? Unilaterally or joint discussion?

 

Ted:     When I have judged with good friends, whose philosophy I know well and who are also experienced judges – for example, Terry or Robbie Bickley - I will discuss dog work in the fourth series, as well as the dog’s relative standing as the dogs perform in the fourth series.  After the last dog has finished, I will take a few moments to myself to organize my thoughts before we start discussing placements.

            When I have judged with people that I do not know well, I remain quiet.  I will go through the process that I described above to come up with what I think the placements should be and ask my co-judge to tell me when they are ready to discuss placements. I will ask my co-judge if he/she has a winner and placements, and ask my co-judge what they are before announcing my thoughts.  I do this to encourage discussion.  If we agree, the task is done.  If not, then we need to go through the process of reaching consensus.

            For the most part, I have found that my co-judge and I agree on the placing dogs, but the issue is what their placements should be.  I put the dogs that are not in contention, and set them aside.  I ask my co-judge which two dogs we should compare first.  We then go through the birds one by one, discuss degree of difficulty, and all of the other factors that we have been discussing through the trial, and decide that one dog is better.  We pick another dog out of our contenders, compare it to our top dog. If it is better, it goes on top.  If it is not we compare it to the dog that we initially compared to the top dog, to see which dog now occupies second place, and so on.  Once we decide on our four places, we look through our JAM pile to see who is best.  This typically does not take long.  Then once we have picked the RJ, I like to compare it to 4th just to make sure that we have done our work properly. 

            Rarely has this process taken me more than 30 minutes to accomplish – but, that is because I have been working throughout the trial on placements – I simply didn’t know the identity of the dogs I would be placing.

            If I am judging with someone that is less experienced (but wants to learn), then I will be less apt to express my opinions at the beginning of the process.  Rather, I will ask my co-judge why he preferred Dog A to Dog B, or what he thought of Dog C’s work on the money bird in the Fourth Series.  I try to employ a gentle Socratic method to help my co-judge articulate what his beliefs are and whether those beliefs are grounded in the Rule Book.

 

Dennis: You are saying that, as I do, you actually begin the Placement Process even before the trial is over, for example while dogs are running the last series water marks?

 

Ted: I actually start even before that. I prefer to judge the Open, because I like watching the handlers who can extract the most out of their dogs.  With a 70+ dog Open, this means Friday is spent on land marks, Saturday is spent on land blind and water blind, and Sunday will be spent on water marks.  By that time, I have watched three series, I have a pretty good body of work to evaluate.  Saturday night, after dinner, I like to go through the remaining sheets and look closely at the dogs one by one.  This is when the notes that I have written on the judging sheets really come into play.  When we start running marks on Sunday, I have a pretty good feel for the relative ranking of the dogs.

            Let’s say we have eight dogs in the fourth series.  Dog 1 runs. It gets the birds cleanly.  It is the winner at the moment.  Dog 2 runs.  It handles.  Dog 1 and 2 have similar work.  I put Dog 2 behind Dog 1 in my book.  Dog 3 runs.  It has similar work to Dogs 1 and 2, but has a moderate hunt on one of the birds.  I put Dog 3 between Dogs 1 and 2. And so on.  I try to do a rough ranking as the dogs are running.  In the fourth series, with water marks – and unlike many judges, I want to watch a dog swim – I have plenty of time to reflect on the dogs’ comparative performance.  I find that by ranking dogs as the test progresses, I am able to save time, and get everyone on the road home sooner.  That’s the mechanical process I use.

            One thing that really helps me in the mechanical process of evaluation is the Houser template system which I described last issue. I am very meticulous in both my initial diagram and in my notes for each dog.  It is a lot of work on the front end, but that work pays off at the end.  When I start really looking at the dogs individually and comparing the dogs to one another, one by one, (typically when looking at the callbacks to the fourth series), instead of in groups, I can overlay several dogs at once over the template. It makes it visually easy to compare the dogs to one another.

 

Terry: Like most judges I try to get a feel of my co-judge’s thoughts as the trial progresses. Since we have already done callbacks together we both have some indication of what we like – or dislike – and I have a general feel of what dogs I believe are near the top and deserving a placement.

            Newer judges tend to struggle processing all the information available at the end of the trial including how to sort through marks versus blinds and determining the winner. I like to let my co-judge make their own determination of placements without influencing them and then proceed with our discussions. I will leave them alone as they evaluate dog work.

            When I have an assignment with an experienced judge I usually begin the process with a back-and-forth comparison of results. There is little chance of unduly influencing an experienced judge and I really enjoy a discussion of relative dog work.

            My method of evaluating dog work and determining callbacks and placements is constantly evolving. When the trial is over and decisions are needed I refer to my judge’s sheets and notes as a primary reference. I typically review dog work as contestants are running and make initial comparisons.

            Normally there are dogs that haven’t done anything outstanding throughout the trial and I place them in a lower tier as they finish their marks. If any dog has outstanding work throughout the trial I place them in the upper tier of dogs. As dogs complete the trial I start my sorting process. By the time the last dog has run I have generally sorted dogs into groups or tiers.

 

Comparing Retrieves - Marks versus Blinds

Dennis: The Rule Book talks about comparing dogs series by series. While those comparisons are often relatively easy for one bird (Dog A was better, the same or poorer than Dog B) a real challenge crops up when you start comparing marks versus blind performance. Dog A has a good mark and a weak blind and Dog B has a marginal mark and a superb blind.

 

Ted: The conceptual process to sort this out is more difficult to explain.  At the most basic level, I compare the dogs to one another, bird by bird.  In a typical trial, I will have a land triple (3 birds), a double land blind (2 birds), a water blind (1 bird), and a water triple (3 birds).  So, if I just evaluate dogs on a bird by bird basis, I am looking at six marks and three blinds.  Mathematically, marks will count twice as much as blinds, and I will have honored the adage that “marks are of primary importance.”

 

Terry: After all dogs have run I then look at all details of the trial to determine my winner. I’m from the school of judges that looks at the entire trial to decide placements. Both marks and blinds are part of my evaluation. I typically look at a difficult mark or blind and score the best dog higher. This is often where the best dog separates themselves from the others. Sometimes we as judges are lucky and one dog is way ahead of the field of dogs. In this case determining the winner is quite easy. Other times the dogs are knotted somewhat closer. I then go on a bird-by-bird, blind-by-blind comparison. The comparison usually results in some differences in dogs and eventually a decision on placements. As the lower placements are made the dogs typically have bigger hunts, poorer blinds, or some combination of the two. Often this is a more difficult situation for me.

            In almost all judging situations I have found the general group of dogs deserving a placement is fairly easy. The specific determination of which dogs receives first, second, etc. can be more of a struggle depending on the results of the test. Anyone who has judged often has faced the situation where two dogs are very close. At that point I would look to see if one dog had better work on a particularly tough mark or blind. If all else fails and the dogs are still closely knotted I would pick the dog with the best marks. I go with that “gut” instinct. Luckily, this has seldom happened to me.

 

Ted:  Yes, the process is rarely simple. It is easy to say that the dog with the “best” marks wins! It is harder to determine what the best marks are.  Not all marks are created equal, and some blinds are more difficult than some marks (for example, the water blind I described earlier as opposed to a short giveaway flyer). So, I think one needs to consider degree of difficulty in evaluating performance.  For example, Terry and I judged an Open at Hank McNeil’s property, where we set up a magnificent land blind (with a small piece of water up front).  It was the best land blind I have ever constructed –  a joy to watch the dogs run – and it was a bear.  That blind was the hardest bird in our trial.  I know that I gave great weight to that bird in our deliberations, because it was so difficult. 

            I can recall another Open that I judged in Northern California, where we had a very hard water blind.  Long approach, skinny water entry, over a little point, then a big swim in cold water.  I remember that Bill Sargenti won with Lily, but I can’t recall anything she did in that trial.  I also remember Jane Patopea with Merlyn.  I can still see that water blind.  It was magnificent.  I don’t remember what Jane got – 3rd or 4th – I think, but I remember that she got that placement on the basis of her water blind.

            In those two instances, I considered a blind to be more significant than any mark because of the difficulty associated with that blind – and my placements reflected that consideration. Also, there was something remarkable about the winning dogs work on a very difficult blind that I believed deserved recognition.  I wanted to reward those dogs because even though “marks are of primary importance,” the Rule Book tells us that we need to consider natural and trained abilities.  I think that the dog that wins an Open should be a well-rounded retriever.

            In the same vein - but less controversially, I suspect – I will weigh a very hard mark more than a less difficult bird.  I will give a hard flyer particular significance – for example the left hand pheasant flyer in the Fifth Series of the 2013 National Amateur - because of the degree of difficulty for that bird (huge flights, pheasant, difficult terrain).

            The more I judge, the more I want the dog that wins to be something special, remarkable for that weekend.  I think I accomplish that by giving the dog extra credit for an especially tough bird, be it a mark or a blind.

 

Evaluating the Total package

Dennis: Let’s add to the bird versus bird and the mark versus blind comparison. One of the things that some of the best judges do, at least in my view, is not fall back on “marking is of primary importance” as a crutch to simplify their job. Those judges see a dog with 6 near perfect marks and they don’t look at much else such as ordinary or even poor blinds, style, or other behavior. Where does the best “all-around dog” come into play and the one that is the “most pleasing”? In other words, how do you factor in the Sletten factor?

 

Terry: I try to determine which dog has an overall performance which is more pleasing to me. That dog would place higher. If the dogs are locked tight for a placement I revert back to the statement “marks are of primary importance” and will award the dog with better marks the higher placement.

            Now-on to my personal evolution as a judge. I think to the Rule Book and the idea of a “pleasing performance” by a dog. I like stylish dogs. Those are the dogs I would choose as a sire or dam and the type of puppy I would like to own. Style has become more important to me and I will make additional notes on a dog’s style as they are running. Poor style will result in a lower score, and conversely, exceptional style will result in a higher score.

            I also think to the Rule Book discussion on pin-point marking. A dog which has exhibited pin-point marking throughout the entire trial has clearly exhibited superior performance. I’m not so sure a dog with extraordinary style and a tiny hunt in the area of the fall hasn’t demonstrated a similarly superior performance. Do we penalize these dogs because they a very fast and disturb slightly more cover? I think not-at least for me personally. This is a roundabout way of saying I agree with Paul Sletton’s analysis.

 

Ted: As I go forward, I will give more weight to style and courage in my judging.  This change is a result of our work in the High Roller Article, my reflection on what Paul Sletten wrote, and the Rule Book section that confirmed Paul’s thoughts, which began our discussion. For example, rather than using style as a tie breaker for two dogs with similar marks, I may use style to elevate a dog with inferior marks above a dog with superior marks, but inferior style.  I may also adjust a dog’s placement downwards on the basis of style alone.  Just as I have been moving to drawing pictures to inform handlers of my expectations, I am considering in a further effort at transparency making an announcement to handlers before we begin that style will play a significant role in callbacks and placements.  As Terry would say, my approach is evolving.  

 

Terry: Never forget that everything we do in judging and evaluating dogs is relative to the work of the other dogs. As quoted earlier by Ted our Rule Book states the following: “It is recommended that a Judge should have clearly in mind, and for each test, precisely what type of performance he expects, since such work will merit a high rating in his records.” A performance pleasing to the eye will carry an increasing amount of importance to me in the future.

 

Does the dog with the straightest lines always win?

Dennis: Judy Rasmuson and I did an article on judging the Derby last year where we emphasized where the dog starts to hunt, what area he hunts and how he hunts is far more important than how the dog gets there. Despite the Rule Book which talks extensively about the area of fall and relatively little about the line to the mark, we often hear, “the dog with the straight lines wins”.

 

Ted:  I have never had it occur when judging, but yes, I am certain that someday my co-judge will say “Lardy says that the dog with the straight lines wins” – referring of course to the Lardy training video, in which the announcer (not Mike) says “the dog with the straight lines wins.” I know that I hear it all the time in the gallery.

 

Dennis: What will you say in response and how do you factor lines in the final evaluation?

 

Ted:     My response would be that the quality of a mark is determined by more than the line that the dog took to the mark.  In evaluating a mark, I would consider:

            Whether the dog “gives definite indication of memory, and of his marking ability, at or after delivery of a first bird, by aligning himself toward, or looking eagerly in the  exact direction of an un-retrieved fall,” AKC Rule Book, page 49.

            Whether a dog “recognizes the depth of the area of the fall,” AKC Rule Book, page 48, by checking down, dropping its head, or cracking its tail, when it reaches the area of the fall;

            Whether the dog “quickly and systematically hunts it out,” AKC Rule Book, page 48.

            All of the above provide the observant judge with valuable information on the quality of a dog’s mark.  (As an aside, I would also say that the above also demonstrates the quality and intensity of thought that the original authors brought to our Rule Book.) I still remember during one of my early judging assignments, my co-judge Vickie Lamb, telling me that she always looks to see which direction a dog first turns when it approaches the area of the fall – towards or away from the mark – as an indication of the quality of the dog’s mark.  I think it is easy to judge a line; it is far more difficult to judge the quality of a mark.  The former takes little dog knowledge; the latter requires a sophisticated understanding of dog behavior.

            So, I would say the dog with the best marks wins – unless another dog with inferior marks, had superior style – or the hardest test in the field trial was a blind.  Or in keeping with Paul Sletten’s comments – the dog that most resembles my idea of the perfect dog wins.

 

Dennis: Terry do you have anything to add?

 

Terry: I try to evaluate a dog’s mark by watching the marking process from beginning to end. Often you see a dog line up for a difficult mark with extremely positive intent. Their body language practically says “I know where it is, cut me loose.” The exact line from the mat to the bird isn’t very significant if the dog knows where the birds have fallen.

            Other times the dog has little idea of where the bird has fallen. The handler tries to encourage the dog to get to the area. If the dog wanders throughout the field in a haphazard manner and stumbles into the bird there is less indication of a quality mark. For me, this dog is scored lower in both callbacks and placements.

            I recently judged a trial with substantial terrain. It was very tough to get to one of the birds. One dog in particular negotiated all the hazards and stepped on the bird. Most dogs found an easier path to the bird by avoiding the terrain, going in to the open field, then winding the bird/gun station from the back side. All these dogs were called back to the next series, but I made notes on the best job and scored this dog higher. Some would argue that I just judged the line to the bird. In my opinion, I simply scored the best mark higher than the poorer marks.

 

Final Decisions and Words

Dennis: The Rule Book gives ample opportunity for differences of opinion. We all hope however that there is some consistency in recognizing superior work from week to week. On occasion we hear of truly opposite views and co-judges at an impasse. Sometimes “politics” is suspected but often it is just a difference in interpretation of what is a better job. How do you deal with that?

 

Terry: Divergent opinions are a wonderful part of the judging process. Our rule Book rates faults as minor, moderate, and major. While major faults, such as breaking in an all-age stake require elimination, many faults are somewhat subjective in nature and require interpretation by each individual judge. The original authors of our rule Book chose to give us considerable leeway in the way we view faults. Given this, there is room for an honest difference in opinion. Relatively minor faults may, in summation, result in a moderate fault or even elimination. This is the exceptional nature of the flexibility in our rules.

            As a judge it is my responsibility to follow and not abuse the fundamental nature of our rule Book. There are no doubt “politics” involved in judging at times but it is my hope that this is mostly a different interpretation and evaluation of dog work. The best and brightest judges ignore personal prejudice and simply evaluate dog work. In the long run, our sport benefits from selecting the best dog work on a given weekend.

            Specifically, if I felt there were politics involved with my co-judge’s placements I would ask that we evaluate each dog bird by bird. This usually makes the political decision difficult to defend and ultimately results in awarding the dog with the correct placement. Somehow, someway, the judges must determine a winner - and make other placements - with complete agreement. Some weekends this is easy, other times difficult, but it must always be done.

 

Ted:  With one notable exception (which I have unsuccessfully tried to forget), I have enjoyed the give and take that comes with working with another judge.  When you work with a co-judge, you are exposed to a different perspective, and I think that it makes you re-evaluate your belief – which I think is good.  For example, I have judged three times with Terry and although we mostly see eye to eye, we still have some differences, and after each of our judging assignments, I have re-considered and changed how I judge.

            Because this article is driven around our individual process, my comments revolve around my individual procedures.  But, of course, judging is a collaborative process – or at least, should be.  I think that when you get two people who are knowledgeable about dogs, passionate about our sport, and committed to providing the dogs and participants with the best product possible, two heads are better than one.  It is why, frankly, I prefer to judge with more knowledgeable judges rather than novices.   When you judge with someone that has been around the block, you can almost take for granted important, but ministerial matters (TIME MANAGEMENT), and focus on setting up the best tests possible.

            One of the great things about judging a National is that you not only have two experienced judges working with you, but experienced marshals, gun captains, etc.  When people are working together for a common goal, I think that the result is greater than the sum of the parts.

 

Dennis: The Retriever News recently analyzed the pool of current judges, noting the number of experienced judges is relatively small compared to the number of events and judges required. In contrast to the obvious benefits of having two very experienced judges working together, it was suggested that it would be beneficial to carefully pair experienced judges with relatively newer judges. I am personally inclined to go that way in the future. Further comments?

 

Ted:   I thought that the article in the News was very thought provoking and timely. As I mentioned earlier, I prefer to judge the Open, and I do not think that a large Open is the place to break in an inexperienced judge.  In that setting, I still think the club and contestants are better off with two experienced hands.  However, I also think that setting is a good place for an apprentice to participate.  I seem to be in the minority in believing that we need to promote our apprentice program. To me, the apprentice program offers the best of both worlds – the club has two experienced judges for its Open, and a newcomer has an opportunity to observe the process without the pressure of driving the process.  I have three judging assignments this year.  I have an apprentice lined up for one, and am seeking one for the other two.

            In all candor, I need to say that I prefer to judge with friends, and not with inexperienced strangers.  Judging is very hard work – if you are committed to doing it well.  And if you have to travel any distance to judge an Open, you are dedicating five days to the endeavor.  If I am going to spend that much time away from home, I prefer I do so in the company of friends.  But, I recognize the need to mentor other individuals and will do my part in the mentoring process.

 

Terry: The only way to gain experience in judging is by actually judging. The internet and gallery gossip won’t make you a better judge. I appreciate all the mentoring and words of wisdom I have received as part of my judging experience. I’m a believer in the pay it forward concept so I would embrace the opportunity to share my thoughts and ideas with newer judges as they groom their own opinions.

            Again, this is my opinion, but pairing an experienced judge with a new judge in a large Open is difficult on both judges and the retriever club. It is better for judges to learn the process of judging in minor stakes and then move on to the Amateur stake. Large Opens deserve two experienced judges. They are difficult and something always goes wrong. The wisdom gained from moving through judging minor stakes and on to major stakes is invaluable to new judges. Many new judges immediately seek all-age points but they can really benefit from line-time experience in the minor stakes.

            I much prefer judging a large Open with another experienced judge and enjoy judging with someone I know. To me, this is also a great opportunity to add an apprentice judge. What better place to discuss rules and learn the nuances of setups, callbacks, and eventual placements? I realize this is somewhat contrary to the Retriever News suggestions about adding new judges but in my opinion the minor stakes, small Amateurs and Opens are the best place to gain judging experience.

 

Back to Top