JUDGING PROCESS — MARKS VERSUS BLINDS – PART 1 – THE CALLBACKS PROCESS

by Ted Shih, Denver CO and Terry Rotschafer, Worthington MN

 

 

IT'S SLETTEN'S FAULT

Ted:   I have been pondering Paul Sletten’s comments to our article on High Rollers in the Fall issue of Retrievers ONLINE:

“An obvious concern when handling and judging a High Roller is missing a piece of cover because you have time for one whistle and not three. Perhaps people could be judging more on style and the type of dog they want to watch or to breed. Just because a dog has one hunt and some other dog was perfect, doesn’t mean the perfect dog has to win! Perhaps the dog that showed more heart and courage and style is the dog we would rather own, breed to, etc. Every judge should have an idea of what their perfect dog would look like and then set up tests so those types of dogs do well at their trial.”

        When I went looking through the Rule Book, I found that Paul’s suggestion was clearly described there:

“It is recommended that a Judge should have clearly in mind, and for each test, precisely what type of performance he expects, since such work will merit a high rating in his records. Then he should observe, and record, in what respects and to what degrees the performances by individual dogs have either exceeded or fallen short of that previously established “par," in each test. Hence, when the stake is completed, several Judges will arrive at their final decision about placings on the basis of which dog, relatively, did better work than another in each of the several series.”

AKC Rule Book, page 47 (emphasis added). (Note, Rule Book citations in this article are AKC but CKC is similar in most cases.)

        That, in turn, made me think about what the Rule Book says about callbacks and placements, and the process I use when I am judging to call back and place dogs.

        Here is what the Rule Book says about the kind of dog that should place and win:

The function of a Non-Slip Retriever is to seek and retrieve “fallen” game when ordered to do so. He should sit quietly on line or in the blind, walk at heel, or assume any station designated by his handler until sent to retrieve. When ordered, a dog should retrieve quickly and briskly without unduly disturbing too much ground, and should deliver tenderly to hand. He should then await further orders. Accurate marking is of primary importance. A dog which marks the fall of a bird, uses the wind, follows a strong cripple, and will take direction from his handler is of great value.

Rule Book, page 26, 2. (Emphasis added). Most conversations around judging, callbacks, and placements revolve around the phrase “accurate marking is of primary importance.” However, I think that the focus on that phrase is misplaced. And, I think that the Rule Book makes it clear that we are looking for more than just accurate marking in a winning dog. I think Paul Sletten hit the nail on the head.

 

Terry:    When discussing callbacks, people often focus on that phrase. However, they seem to place little attention to the words that follow that phrase in the Rule Book.

Ability to “mark" does not necessarily imply “pin-pointing the fall.” A dog that misses the “fall” on the first cast, but recognizes the depth of the “area of the fall,” stays in it, then quickly and systematically “hunts-it-out,” has done both a creditable and an intelligent job of marking. Such work should not be appreciably out-scored by the dog that “finds” or “pinpoints” on his first cast. However, a dog which consistently, i.e., during an entire stake, marks his birds in a closer area, hence, more accurately than another dog, should be judged accordingly. All things are relative, and, conceivably, such differences in markings alone might be sufficient to determine the final placings in a particular stake.

Many judges tend to look at accurate marking as simply “pin-pointing the fall.” But, marking is more than that. I think an exploration of how judges – and in particular, you and I – make decisions about callbacks and placements would be fun.

 

TIME IS A PRECIOUS COMMODITY – USE IT WISELY

Ted:   In a trial, time is a precious commodity. Judges that use it wisely can have tough tests and generous callbacks. Those that squander it, typically resort to pencil whipping and elimination blinds to mask their sins.

The Rule Book acknowledges the relationship between time and callbacks:

To the extent that time permits, Judges should be generous in their “callbacks” for additional series. No dog should be eliminated from further competition unless it is the consensus of the Judges that it would be impossible for him to “place” in the stake, even though his work in all succeeding series was perfect.

        Page 46, 12 (italics in original, underlining added).

Field trials are a sport, and in my opinion, should be fun. So, to the extent possible, I want to be generous in my callbacks. But, that means that, as a judge, I must manage my time effectively.

Terry:   I also strongly believe in time management and its influence on callbacks. Pre-trial preparation by the judges and an ambitious setup crew allow for more time and therefore more generous callbacks. It takes both. A prepared judge without workers to throw birds still results in unnecessary delays, late starts, and possibly less generous callbacks.

 

KEEPING TRACK – THE MECHANICS OF RECORDING DOG PERFORMANCE

Ted:   Before we get into the conceptual process each of us uses in evaluating dog work and in making decisions on callbacks and placements, I think it makes some sense for each of us to describe our respective note taking system.

 

Terry: The system I use is relatively simple. I take numerous detailed notes as the dog is running. I note large hunt, small hunt, pickup, handle, style (+ or -), long flyer, SOB (stumbled-on-bird), etc. I like using the Callbacks Sheet which is included when I order judges sheets. I use a grade to score dogs. This is my initial reaction to how the dog performed and I use it as a reminder when doing callbacks. I don’t use my grades as an absolute. They are not used to determine a winner but simply as an initial impression and appraisal of performance. I realize everyone develops their own system. Whichever system used, I think it is important for judges to use some method of tracking work to expedite callbacks. No one wants to sit in the gallery for two hours while you and your co-judge analyze each and every page. There is only so much daylight. Time management can be substantially improved with a scoring system and a little effort.

 

Ted:   I employ a summary system that is similar to yours. I use the tally sheet in the Entry Express Event Catalog. As each dog completes the first series, I put a (é) if I think that the dog’s work was above average. I put a (–) if I think that the dog’s work is average. I put a (ê) if I think the work was below average. Dogs that pickup get a (P). Dogs that handle get a (H). As we will discuss later, I rely heavily upon my summary sheet in evaluating callbacks.

        I use Lee Houser’s system to record the performance of individual dogs. Before we start running dogs, I will make a detailed diagram on a hard plastic template. As each individual dog comes to the line, I will take the clear mylar sheet with that dog’s number on it and place it on the template. I draw cross hairs on the template and on each sheet, so that each dog’s sheet is similarly aligned on the template. I will then use a pencil to draw each dog’s path on its designated overlay. I find that: a) I save time by not having to draw a diagram of the test for each dog; b) I can include more details in my diagram (trees, stumps, change in cover, etc.); c) I can take more detailed notes on dog performance because my diagram is more thorough; and d) when it comes time for placements, I can overlay multiple dogs on the template for a quick visual comparison of dog work.

        I will make detailed notes in short hand if I see something good – or bad – that deserves comment (e.g. GS – Good Style, PS – Poor Style, GIL – Good Initial Line, PIL – Poor Initial Line). I will also note if a dog did well when the conditions were against it (wind, light, etc.). If I see something out of the ordinary, I want to make book on it.

        I don’t grade, either with letters or numbers, as I go along. I find it hard to establish a score as I am judging dogs in the moment.

 

DON’T WAIT UNTIL YOU NEED TO DO CALLBACKS TO DISCUSS YOUR JUDGING PHILOSOPHY WITH YOUR CO-JUDGE

Ted:   I hear of – and have experienced – trials where the judges have spent hours on callbacks and placements. I believe that this occurs because: a) the judges have not discussed their judging philosophies with one another (or have not listened to one another); b) have not discussed dog work as the trial progressed; and c) have (in the case of callbacks) looked at individual dogs rather than clusters of dogs. I am certain we will discuss the third subject later, but let’s start with the first two items.

        If you and your co-judge spend a day together, wandering the field, picking out marks and blinds, you learn a lot about one another’s preferences. You learn a lot about your co-judge in a day (and even more at a National, where you spend a week setting tests). Use that time productively to discuss what you like, don’t like, and how you are going to resolve any differences that you have in judging philosophy. In the same vein, the judges should discuss the dogs as they are running. That way, when it comes to deciding on callbacks, you have already done the bulk of the heavy lifting. I have to think that judges that spend an hour or more on callbacks have simply deferred the heavy lifting to the last minute. It is rare when I judge that I spend more than 10 minutes on callbacks, or 30 minutes on placements. But, that is because I have already devoted multiple days to the process.

 

THE FIRST SERIES CALLBACKS

Ted:   Let’s start by discussing the process to determine which dogs to call back after the first series.

        I am a traditionalist. As we discussed in our previous article, I will configure my tests to conform to entry size, weather, light and terrain. However, given my druthers, I prefer to start with a wide open triple, two birds retired, flyer shot last. So let’s assume that is our first series.

        Given time, I prefer to call back all dogs that have not handled. There are some exceptions, of course. If a dog hunts all over the countryside on a bird and manages to stumble across the bird in its aimless wanderings, I would view that dog as not having a mark, and consider its work as a failure. If a dog had wide ranging hunts on multiple birds, I would consider that dog to have “unduly disturbed too much ground,” Rule Book, page 40, and to have failed as well. In addition, in the appropriate situation, I would bring back a quick handle.

 

Terry:  I am in complete agreement with your statements. Given appropriate time, terrain and entry size, I also prefer open marks with generous callbacks. Dogs which have multiple hunts which are significantly large or perhaps a gorilla hunt out of the area of the fall are probably on my bubble list. They may or may not be called back depending on the work of other dogs and the amount of time available.

        Per the Rule Book and my earlier quote I don’t eliminate dogs with insignificant hunts in the area of the fall. It has become apparent that some judges view very minor hunts as a failure to pinpoint the mark and justify elimination based on this premise. Determining the area of the fall and the size of the hunt is subjective and often varies from one judge to the next. I would choose not to eliminate these dogs.

        The topic of a handle on a mark is interesting. As a practical matter, in most current field trials the handler often lets the dog put on a significant hunt hoping the dog will retrieve the bird, then after a large hunt decides to handle the dog to the bird. To me, this equates to two errors, first a large hunt, second a handle. In this type of situation, the dog is generally on the bubble and, most often, eliminated. A true quick handle would be treated more favourably than a gorilla hunt and handle.

 

Ted:   On page 50, the Rule Book states as follows:

Dogs which disturb cover unnecessarily, clearly well out of the area of the “fall,” either by not going directly to that area, or by leaving it, even though they eventually find the bird without being handled, should be penalized more severely than those handled quickly and obediently to it.

        I think this means that judges need to penalize wide ranging hunts (see discussion above) and handles that are not “quick” and “obedient.” As a practical matter, this means that very few handles are eligible for return. Why do I say this? Well, if your dog gets to the area of the fall, you are unlikely to handle, knowing that the general judging community punishes handles severely – regardless of what the Rule Book says. So, once your dog gets to the area of the fall, you let it hunt, and hunt, and hunt, and by the time you handle – if you are able to accomplish it “quickly and obediently” your dog has likely disturbed cover well out of the area of the fall. Alternatively, if your dog runs to 12 o’clock, when the bird is at 3 o’clock, and is far from the area of the fall, I think your dog has shown no mark at all.

        In my opinion, there are two scenarios where a quick handle might play: a) a pheasant flyer that has fallen well clear of the majority of the other birds, the dog makes an initial cast just short of the bird, then gets trapped in the scent of prior falls; b) a nasty short retired bird, where the dog is building momentum, as it is passing through the area of the fall. In the first, the dog has shown a mark, and is making a reasonable hunt, given the scenting conditions. In the second, the dog may or may not have a mark, but it has run directly to the area of the fall, has not disturbed cover unnecessarily and got the bird. I would view a handle as a smart and productive move in both situations and probably bring back the dog.

        There is a school of thought that because “call backs should be generous,” if time allows, and a number of dogs have huge hunts and/or handles, all the dogs should be brought back. I have not been faced with this situation, but my initial reaction is that I would not call back all of the dogs. I would have a hard time telling myself that a dog that failed one or more marks would still be in contention for a placement.

 

Don’t waste time on callbacks.

 

Ted:   The less time you spend on callbacks, the more time you have to test dogs. As we discussed earlier one way to save time is to discuss dog performance as the dogs are running. If you and your co-judge are discussing dog work as the trial progresses, by the time the test is complete, you should have a pretty good idea of which dogs are in and which dogs are out. That means that your discussion will focus on the dogs “on the bubble.”

 

Terry:   I agree. I generally look at callbacks in this manner: In, Out, or On the Bubble. The Bubble dogs are the ones I want to discuss with my co-judge. I feel that the judges must be in agreement to eliminate these dogs. If not, they should be called to the next series. Do not waste large amounts of time on disagreements. It is often far quicker to run the dog on the next setup. Again, time management is important to your trial.

        My general feeling on callbacks is to be as generous as time and results allow.

 

Ted:   On page 43, the Rule Book states: “No dog should be eliminated from further competition unless it is the consensus of the Judges …” Both judges must agree before a dog may be eliminated from competition.

        I find that when you look at the dogs “on the bubble” they tend to cluster in groups. For example, a group of dogs will have had a huge hunt on the long retired bird. Another group of dogs may have broken down in the flyer scent, and gotten into a big hunt out of the area of the fall on a short retired. Don’t look at individual dogs in each group. The differences are likely to be insignificant. Instead, consider recalling or dropping the entire group. If you can’t decide, err on the side of generosity. Like you, I find it is easier – and more productive – to simply test more dogs than spend time arguing about whether individual dogs should return.

        Finally, you and I both copied a system we saw Lou Vreeland use at the Omaha Field Trial. We prepare callback sheets in advance with the numbers for the dogs entered (say 1-60). When we have decided on callbacks, we fill in the blank – Callbacks to ___Series, and we circle the numbers of the dogs that are returning. I like the uniformity of Lou’s method, and I think it saves time.

 

SECOND/THIRD SERIES CALL BACKS – MARKS AND BLINDS

Ted:  When you get to callbacks from the second and third series, I think things get more difficult. Time is more important. In addition, you must start evaluating the significance of marks and blinds.

        The Rule Book divides abilities into two categories: a) Natural Abilities and b) Trained Abilities.

The Judges must judge the dogs for (a) their natural abilities including their memory, intelligence, attention, nose, courage, perseverance and style, and (b) their abilities acquired through training, including steadiness, control, response to direction, and delivery.

        Rule Book, page 31, 22 (Emphasis added)

 

Philosophies of evaluating marks and blinds in determining callbacks

        When it comes to evaluating marks and blinds, I find that judges come in the following flavours:

      Elimination blinds, where the blinds are judged on the ability of dog and handler to navigate one or two criteria (a keyhole, a poison bird, etc.) If you pass the criteria, you move on. If you don’t, you and your dog are eliminated – regardless of the quality of your marks.

      The blinds are a mere formality if you have good marks. That is, if a dog has good marks, it doesn’t matter what it does, as long as it gets the birds.

      Marks and blinds are judged in combination to obtain callbacks.

 

Terry:   My experience is similar. When scoring dogs for purposes of callbacks it is interesting to note the differences in judging philosophies.

        One group of judges has the philosophy that the marks are of primary importance and therefore view blinds as a Pass/Fail mechanism to reduce dog numbers. This group often looks at placing dogs primarily on their marks. They therefore run blinds as a method of eliminating dogs so that marks can produce the eventual winner. A better land or water blind by the dog/contestant team may not have as much impact on callbacks and placements as a slightly better set of marks.

        The other group of judges elects to consider all series as part of the callback and placement process. Each series is evaluated and is considered to move on to the next. I consider myself to be part of this group. I would look at all previous work when determining callbacks.

 

Ted:    I think that the first category of judges are ignoring what you mentioned in our discussion on High Rollers – the need to judge blinds in their entirety: “In general, the performance in the test should be considered in its entirety; an occasional failure to take and hold a direction may be considered a minor fault, if offset by several other very good responses.” See page 54. Emphasis added.

        When you establish a blind with one or two critical features where a failure to negotiate a specific hazard results in elimination, you are also electing to ignore the entirety of the blind. In addition, with an elimination blind, “marking is not of primary importance” because if you fail one critical point, the quality of your marks is meaningless. I disagree with this first approach, but we both know that there are a significant number of judges who do not and are, in fact, practitioners of this approach. I think the prevalence of elimination blinds in modern field trials is a function of: a) an insufficiently difficult first series; b) lack of confidence in the judges’ ability to regain control of the field trial without resorting to gimmicks; c) poor time management; or d) a combination of all three. This may be something we should explore in another discussion.

        I believe that the second category of judges are ignoring what I quoted earlier – The Rule Book tells us that we are to evaluate natural and trained abilities – not just one portion of natural abilities – marking. Marking may be of “primary” importance, but it is not of “exclusive” importance. Increasing entry sizes, limited time, and better dogs have made this approach less popular, however. Moreover, in my experience, more judges are concerned with getting numbers down, than they are in having “generous” callbacks.

        As you might guess, I belong to the third category. I evaluate marks and blinds in combination. I believe that great marks grant a dog grace but not absolution. That is, I am willing to forgive – temporarily – sub-par blinds from a dog with great marks. However, great marks do not grant absolution for abysmal performance. A dog with poor style – or one that runs around with no sense of purpose – or a dog with repeated failure to take and hold direction will be dropped regardless of the quality of its marks. It is my way of honouring the adage that marks are of primary importance.

 

Water Blinds and Diagrams

Terry:   I view water blinds much the same as land blinds. Make sure to cover the basics. The handler should be able to see the dog, the dog should be able to see the handler, dogs must be able to hear whistles, etc.

        Quite often the water blind has more distinct obstacles which must be achieved. Take the case of a water blind in which the line to the blind is across a point. Many judges will give no instructions on the blind. They let contestants run the blind however they choose, then decide on callbacks. At times, the judges will give instructions to the handler such as “the dog must have four dry feet on the point.” This is basically an absolute. Once instructions such as this are given it implies that any dog without four dry feet is automatically eliminated. I think anytime there is a question about whether an obstacle is important, the judges should let the contestant know their preferences. I realize some will disagree. I have run many trials under both types of judges and personally don’t have a problem with either. It is simply my preference to let contestants have an idea of my expectations rather than do the “surprise” callbacks … you missed the obstacle therefore you’re dropped. Again, let me reiterate. Some parts of a blind should be obvious to all and should not require instructions. At times there are parts of a blind in a gray area. If you have a mandatory requirement that is not obvious to contestants, I see no harm in informing them of your wishes.

        Recently, some judges have added a diagram of their best or preferred line to the blind and posted the diagram in the holding blind. Again, I have no issues with this and consider this fair to all contestants when the line to the blind is in question.

 

Ted:   I am on record as being in favour of a diagram which shows the “ideal line” to the blind. I want competitors to know what I want in a blind. If I am going to follow the Sletten/ Rule Book approach, that is – if I am going to have “clearly in mind … precisely what type of performance [I] expect” then I am not going to keep my expectations a secret.

        In addition to including a diagram of what I consider to be the “ideal” line to the blind, I also include the following language so that competitors understand the process that I use in calculating callbacks:

The diagram below illustrates the “ideal” line to the blind. Callbacks will be based on “relative” performance on this blind and “cumulative” performance throughout this stake. However, contestants should not be surprised when failure to attempt to negotiate the line as illustrated results in elimination.

        If I go to the trouble to draw a diagram that shows you what I want, and you – as a competitor – choose to ignore my diagram, don’t be surprised if I drop you on a blind. A blind is a test of control – with the parameters that the judges establish. Ignore those parameters, and I have no basis for evaluating the control you have over your dog. That typically results in elimination when I judge.

 

Judging Cumulative Dog Work

Ted:   I find that the deeper I get into a trial, the less generous I become. I know that sounds crass. It really isn’t. Let me explain. When a trial starts, I have relatively little data on each dog and on the overall performance of the field. As a trial progresses, I know more about the merits of each dog and of the strength of the field. Consequently, I have a better feel for the kind of performance it will take to place – or win. Because I have more data, I can be more discriminating in my judgment. As a trial progresses, callbacks become less generous because there are fewer dogs in contention.

        By the time the third series of a trial is completed, I have a body of work to consider. In deciding who gets to run the fourth series, I can review a pattern of performance. I use the catalogue as my summary format. Generally speaking, I do not look at my sheets, unless my co-judge wants to discuss calling back – or dropping – a dog or dogs, or it is time to discuss placements. For example, if after three series, I might have in my catalogue:

Dog A: é, é, é. It’s easy for me, the dog gets called back. The dog has had superior performance throughout the trial.

Dog B: ê, ê, ê. It’s also easy for me; the dog has had inferior performance throughout the trial. I have no desire to look at it any further.

Dog C: –, –, –  . This dog has done nothing remarkable, but it has done nothing wrong either. It gets called back.

Dog D: é, ê, ê. I bring this dog back, particularly if the first series had few dogs that hit the marks hard. An example of how great marks bring grace.

Dog E: ê, ê, – . Gone, this dog has not done anything to distinguish itself.

Dog F: –, ê, ê. Gone, this dog has not done anything to distinguish itself.

Dog G: ê, é, é or even H, é, é. I bring this dog back, because after an initial bobble, the dog has shown superior work.

        I have used the examples above to give our readers an example of my thought process. However, because I have refined this process over the years, it takes very little time for me to go through my progressions. I do it almost subconsciously now.

        To my knowledge, I have never dropped a dog that did well on the last test, regardless of how marginal its work was going into last test. As a competitor, I would be upset and angry, if a judge brought me back, my dog did well on the next series, and I was dropped. I try to adhere to the golden rule. If I hate a practice as a competitor, I try to avoid it as a judge.

 

WHAT TO DO WHEN THE TESTS ARE TOO HARD OR TOO EASY

Terry: Ted, do you consider the relative difficulty of a test when considering callbacks? For example:

      The marks were fairly difficult and a large number of dogs failed.

      The land blind was more challenging than you expected – or the blind was easier than you initially planned.

Do these circumstances result in you adjusting your callbacks? If so, how?

 

Ted:   Yes, I do factor in degree of difficulty – both in callbacks and in placements. This is hard to discuss in the abstract, though.

        If I lay an egg, and the test is too easy. I will bring everybody back. I view the light test as my failure, and will not pencil whip the dogs and make it their failure. I don’t like getting pencil whipped when I compete, and so I don’t pencil whip when I judge. I will simply make the next series harder to compensate for my mistake. So, I don’t think that the test that is too easy really factors into our discussion.

        If my test is too tough, then I will make adjustments.

        For example, if only a handful of dogs get the first series marks cleanly, then I am probably going to call back dogs with one, maybe two monster hunts, and maybe even quick handles – depending on time. This is supposed to be fun for the participants, so if I have the time, I will let people play. However, I think you need to be careful and still impose some standards – if you do not, I think you run the risk of demeaning the work of those dogs that put in a superb job. Moreover, if a dog fails (breaks, eats a bird, returns without a bird), it fails. My generosity on a tough test does not extend to ignoring serious faults.

        In the same vein, if I have a blind that is too tough, I will probably have to bring back dogs that I felt had a sub-par performance. A couple of years ago, I judged an Open. We had excellent but tough land marks and land blinds. In retrospect, the water blind was too hard. It was a long entry, skinny water blind, with a tough cast off of a long but skinny point. We scented the shoreline and the point, and had a wet sluice. Five dogs did the blind in an acceptable fashion. One was exceptional and eventually won. There were probably another six or seven dogs that had really hideous blinds. We called back the five only. And in the fourth series, all five handled.

        In retrospect, I wish that we had called back the other six or seven, to see if any of them could have gotten the marks. So, my process is evolving as I evaluate what I have done well – or poorly.

Terry:   I’ll be brief because we judge in the same manner. If a test is too easy I’ll chalk that up as my mistake and bring back all successful dogs. Rather than nit-pick or pencil dogs it seems easier to bring these dogs back and try again with another test. Tests which are very difficult allow me to be quite generous on callbacks. If the test turns into a “survival” test, I like to bring back as many dogs as possible. Of course, dogs which do the test well are scored higher, but I see no reason why dogs that are less than perfect can’t be called back. By the same token, dogs which completely fail my standards must be dropped to be fair to the successful dogs.

        My judging is also a continuously evolving project. It is often difficult to determine the degree of difficulty for a test until we get started. All of us have been fooled with tests which are more or less difficult than anticipated. I want to be as generous as time and conditions allow.

Editors' Note: In Part 2 of The Judging Process, I will interview Terry and Ted about Picking the Winner at the end of the trial. Stay tuned!   

 

 

Back to Top