Simulator Grading

bob smith

Trusted Contributor
Messages
149
Anybody see the new grading in the 3210 for simulator problems? They are grading like the academy and you can earn “plus points” for demonstrating exceptional ability in “equipment usage” during the problem as well as other categories. Seems like they want every breathing body to make it to the floor now at a minimum.
 

tunwno

Trusted Contributor
FAA
Messages
428
Facility
Mike Monroney Aeronautical Center
Anybody see the new grading in the 3210 for simulator problems? They are grading like the academy and you can earn “plus points” for demonstrating exceptional ability in “equipment usage” during the problem as well as other categories. Seems like they want every breathing body to make it to the floor now at a minimum.
?
 

SCofATC

Member
FAA
Messages
38
Facility
N90 New York Tracon
I am guessing Bob means 3120.4. And I can only find version P online. Where can we find this new verbage? If it makes it easier to get through classroom and simulation to get to the more hands-on training and pass/fail there I would think that is a good thing.
 

32andBelow

Forum Sage
Messages
570
Anybody see the new grading in the 3210 for simulator problems? They are grading like the academy and you can earn “plus points” for demonstrating exceptional ability in “equipment usage” during the problem as well as other categories. Seems like they want every breathing body to make it to the floor now at a minimum.
Which is good cus the lab is nothing at all like real life.
 

mrobs15

Forum Mage
I ♥ pointSixtyFive
Messages
390
Idk where to find it but I did get a CEDAR item saying all simulator and written tests must now use 70% as the passing number.
 
Messages
40
Yeah we got this too. All sim evals, written tests, map tests etc. now can't have a passing score higher than 70%. Now you can have a sep error in a sim eval and would still need to lose 15 more points before it would be a fail. Should result in a lot more people making it to the floor at N90 instead of washing out whole classes in the sims.
 

Stinger

Moderator
Staff member
Messages
1,170
If the problems are graded the exact same, and all the little points are added up and deducted, it will result in more people actually failing in the simulators. I'd be surprised if many facilities have come up with ways to add positive points back in.
16 points for a separation loss, but possible for 5 points (or 4?) to be added back in the separation category. I can only see that happening if there's problems designed to have a separation loss, but the trainee saw the conflict and fixed it.
Only way I can see this working (and the memo should have included it) is to describe how to add positive points. TETRA uses a set number of "goals" that can be accomplished in the problem. The goals include things like: aircraft crossing the threshold, leaving the lateral confines of the airspace, getting airbone as a satellite departure, etc. Each goal is worth something like 2.5 or 3.3 points.

Nationwide, facilities are going to have to rework their problems to get an effective scoring system in place. Or, have someone work the problem in an efficient way and determine what the positive points are going to be. The negative points are easy enough to figure out, but those are going to add up quick...especially on the 100-125% problems.
 

bob smith

Trusted Contributor
Messages
149
If the problems are graded the exact same, and all the little points are added up and deducted, it will result in more people actually failing in the simulators. I'd be surprised if many facilities have come up with ways to add positive points back in.
16 points for a separation loss, but possible for 5 points (or 4?) to be added back in the separation category. I can only see that happening if there's problems designed to have a separation loss, but the trainee saw the conflict and fixed it.
Only way I can see this working (and the memo should have included it) is to describe how to add positive points. TETRA uses a set number of "goals" that can be accomplished in the problem. The goals include things like: aircraft crossing the threshold, leaving the lateral confines of the airspace, getting airbone as a satellite departure, etc. Each goal is worth something like 2.5 or 3.3 points.

Nationwide, facilities are going to have to rework their problems to get an effective scoring system in place. Or, have someone work the problem in an efficient way and determine what the positive points are going to be. The negative points are easy enough to figure out, but those are going to add up quick...especially on the 100-125% problems.
I think it will depend on the facility, which will add a ton of subjectivity to the process. One facility might not care about nitpicking your pointout phraseology while another might knock you off on communication for clicking the pickle twice.
 

Stinger

Moderator
Staff member
Messages
1,170
I think it will depend on the facility, which will add a ton of subjectivity to the process. One facility might not care about nitpicking your pointout phraseology while another might knock you off on communication for clicking the pickle twice.
TETRA runs with a cap on the amount of phraseology points that can be deducted...it's somewhere between 5 and 10. There was also caps on the recent memo that went out to facilities, but I didn't memorize them and don't have the memo in front of me.
And yes, I know someone that was deducted phraseology points for ending transmissions with "thanks" "bye" "see ya" etc...
 

DankVectorz

Forum Sage
FAA
Messages
791
Facility
N90 New York Tracon
I’m ok with being allowed to have a sep error in the lab. We’ve lost too many trainees that showed potential and did really good the whole lab and had just a couple mistakes in the eval. You’re a trainee, mistakes should be expected. The lab should be to demonstrate that you can learn the material etc.
 

vectorwagon

Trusted Contributor
Messages
136
I’m ok with being allowed to have a sep error in the lab. We’ve lost too many trainees that showed potential and did really good the whole lab and had just a couple mistakes in the eval. You’re a trainee, mistakes should be expected. The lab should be to demonstrate that you can learn the material etc.
Yeah I feel like they expect you to run 110% problems flawlessly on your own when you have 16 hrs of experience with no REAL training, and from some guy who last worked traffic 10 yrs ago like someone else mentioned. And that just doesn’t make sense. I will say that 110% in the real world is a whole different ball game then the lab and they want to make sure you are somewhat capable before turning you loose on the real stuff but I support modifying the lab grading somewhat too.
 

Electricfence

Active Member
Messages
80
the simulator problems should have been vetted by the facility. The workforce, both union and management absolutely should be involved in the creation and certification of these lab problems down to and including what percentage of complexity should be acceptable for the eval, which should be in your local training order. if you dont like what is going on, get involved and ask questions and work toward the change.
 

FightingIrish2012

Trusted Contributor
Messages
401
Does any facility other then N90 that you know of have huge washout rates in the simulators? From what I have seen and generally heard almost everywhere they do all they can to pass people. Even when people double fail a eval there is often some reason that comes up to give them a second chance or let them through. I think this is good, there would be nothing worse then washing out on a video game with only 16 or so hours of sim training as people said.
 

32andBelow

Forum Sage
Messages
570
Yeah I feel like they expect you to run 110% problems flawlessly on your own when you have 16 hrs of experience with no REAL training, and from some guy who last worked traffic 10 yrs ago like someone else mentioned. And that just doesn’t make sense. I will say that 110% in the real world is a whole different ball game then the lab and they want to make sure you are somewhat capable before turning you loose on the real stuff but I support modifying the lab grading somewhat too.
Real world is busier but the lab got some guy that is strategically meant to clip within 2.4 miles of every boundary and then violate some ultra obscure part of some obscure LOA.
 

bob smith

Trusted Contributor
Messages
149
I think it’s dumb, so any way to eliminate washing out in the labs is good. If you wanna wash people out in labs keep that to the academy. There shouldn’t be a double jeopardy of going to the academy passing through those labs then moving to bumfuck USA and going through another lab screen.

I believe ZMP has a relatively high washout rate before hitting the floor.
 

jdatc

Trusted Contributor
Messages
192
I think it’s dumb, so any way to eliminate washing out in the labs is good. If you wanna wash people out in labs keep that to the academy. There shouldn’t be a double jeopardy of going to the academy passing through those labs then moving to bumfuck USA and going through another lab screen.

I believe ZMP has a relatively high washout rate before hitting the floor.
Completely disagree. The labs at a facility are the next check point to keep people on the floor from training those that slip through.
 

NavyTrev

Active Member
FAA
Messages
63
Facility
I90 Houston Tracon
Anybody see the new grading in the 3210 for simulator problems? They are grading like the academy and you can earn “plus points” for demonstrating exceptional ability in “equipment usage” during the problem as well as other categories. Seems like they want every breathing body to make it to the floor now at a minimum.
Guy here failed an 80% graded with a 58 on Friday, doesn’t look like it’s all that much of a cake walk.
 
Top