Initial Findings Put Boeing's Software at Center of Ethiopian 737 Crash:
At a high-level briefing at the Federal Aviation Administration on March 28, officials revealed "black box" data from Ethiopian Airlines Flight 302 indicated that the Boeing 737 MAX's flight software had activated an anti-stall feature that pushed the nose of the plane down just moments after takeoff. The preliminary finding officially links Boeing's Maneuvering Characteristics Augmentation System (MCAS) to a second crash within a five-month period. The finding was based on data provided to FAA officials by Ethiopian investigators.
The MCAS was partly blamed for the crash of a Lion Air 737 MAX off Indonesia last October. The software, intended to adjust the aircraft's handling because of aerodynamic changes caused by the 737 MAX's larger turbofan engines and their proximity to the wing, was designed to take input from one of two angle-of-attack (AOA) sensors on the aircraft's nose to determine if the aircraft was in danger of stalling. Faulty sensor data caused the MCAS systems on both the Lion Air and Ethiopian Airlines flights to react as if the aircraft was entering a stall and to push the nose of the aircraft down to gain airspeed.
On March 27, acting FAA Administrator Daniel Ewell told the Senate Commerce, Science, and Transportation Committee's aviation subcommittee that there had been no flight tests of the 737 MAX prior to its certification to determine how pilots would react in the event of an MCAS malfunction. He said that a panel of pilots had reviewed the software in a simulator and determined no additional training was required for 737-rated pilots to fly the 737 MAX.
What follows is from memory from what I've gleaned from reading several news accounts over the past few weeks. I am not a pilot, so take this with the proverbial $unit of salt.
The design of the MAX version of the Boeing 737 used a larger diameter engine so as to improve fuel economy. Because the original 737 was designed to be low to the ground to facilitate boarding (no jetways back then), it required the engines to be mounted forward and higher than in previous models. This introduced a change in the flight dynamics. Adding throttle in certain conditions would cause the plane to "nose up". Because of the shear size of the engine nacelles, this further increased the lift of the nose (more surface exposed at an angle to the air flow). This would cause further lift and would exacerbate the situation. Boeing wanted pilots to be able to fly the MAX without undergoing expensive retraining. How can they make a different aircraft behave like its predecessor, the 737-NG? The solution Boeing came up with was MCAS which — in certain circumstances — was designed to push the plane's nose back down. So much authority was provided to this adjustment, and its repeated application in some cases, that it could lead to driving the plane downward in spite of the pilot's efforts to maintain level flight. Complicating matters, there was no mention of MCAS in any of Boeing's training materials: pilots were not even aware it was there.
It would be easy to "armchair quarterback" Boeing's decisions. The airline industry was transitioning from its hub-and-spoke system (which favored larger planes) to having a greater number of direct (no layover) flights which favored smaller aircraft. In the meantime Airbus had come out with a new model of more efficient aircraft which fit this flight profile. Boeing could have come up with a clean-slate design for a new aircraft, but that would require several years from design to construction to certification. They elected to modify the 737, instead. As long as it was sufficiently similar (I'm waving my hands around a bit here), it could be sold based on the certification of its earlier models. So, they decided to modify the 737... but not too much so as to avoid the time-consuming recertification process.
I've heard it said, "The longest distance between two points is a shortcut." It is sad that this shortcut appears to have been responsible for two flights crashing shortly after takeoff and killing nearly 350 people.
(Score: 1, Insightful) by Anonymous Coward on Saturday March 30 2019, @08:19PM (1 child)
Everyone is an expert at it.
(Score: 0) by Anonymous Coward on Saturday March 30 2019, @10:33PM
Ladies and gentlemen, OP proves his point ... well, not necessarily the software part.
(Score: 1, Informative) by Anonymous Coward on Saturday March 30 2019, @08:21PM (9 children)
My car has a bad sensor that keeps "randomly" activating my antilock breaks, which overrides my attempts to stop the car.
Also, the 737 MAX has about the expected accident rate for a plane with so few flights:
https://i.ibb.co/n138VFj/planecrash.png [i.ibb.co]
(Score: 1, Informative) by Anonymous Coward on Saturday March 30 2019, @10:02PM (2 children)
That's the breaks, sorry to hear that your antilock brakes are broken. [usage nut, sorry, can't help myself]
(Score: 0) by Anonymous Coward on Saturday March 30 2019, @10:19PM (1 child)
Thanks, for some reason I keep making that typo.
On the plus side I am no longer afraid to fly since driving a car with unreliable breaks is far more dangerous.
(Score: 1, Touché) by Anonymous Coward on Saturday March 30 2019, @10:35PM
I hope one day you can find a way to brake that habit.
(Score: 1, Informative) by Anonymous Coward on Sunday March 31 2019, @01:54AM
Airbus A320 had...issues... when it first came out with its software, too. Airbus philosophy then was "Otto Pilot Uber alles" - the autopilot would not disengage if pilot inputs contradicted the autopilot, whereas Boeing's systems then would. This did result in a few crashes early on. One incident in particular was recorded, where the plane was obviously struggling trying to figure out level flight, and crashed into a forest. At the time, there wasn't a way for the pilots to turn off that particular system... I think Airbus adjusted their thinking a little bit after a few crashes...
(Score: 2) by sgleysti on Sunday March 31 2019, @07:29AM (2 children)
I had the same problem on my car. It turned out to be a cracked reluctor ring; I had to replace the axel (half shaft?) for that wheel, since the ring itself could not be removed from the axel.
(Score: 2) by driverless on Sunday March 31 2019, @08:23AM (1 child)
I'm having a bit of trouble following that, could you explain it using a car analogy?
(Score: 1) by Sulla on Sunday March 31 2019, @03:29PM
Not a problem my man. You see the problem with a bad sensor causing ABS failure requiring the replacement of the half-shafts, if we were to use a car analogy, is like the sensor detecting window level failing so your have to replace the driver side door.
Ceterum censeo Sinae esse delendam
(Score: 2) by sonamchauhan on Sunday March 31 2019, @10:10AM
What a nonsensical post.
300 planes and 2 crashes with all passengers killed. Try telling that to your bus company or metro with a straight face.
(Score: 3, Informative) by RS3 on Monday April 01 2019, @04:44PM
Detailed explanation for those who don't know: ABS sensors are usually a (wire) coil around a permanent magnet which is located in close proximity to a reluctor ring (looks like a gear) which is around an axle or wheel hub. As the wheel rotates, the moving reluctor ring teeth create a varying magnetic field strength which induces an electric current in the coil. The coil is connected to the ABS electronic controller.
The symptom you described is usually steel dust particles stuck to the the magnet. They tend to form a magnetic short-circuit so that there is much less current induced in the coil, so the ABS system gets very little signal, so it thinks you have a wheel lockup and it kicks in. I suspect a lot of perfectly good ABS sensors get replaced for this reason.
One of my vehicles has a fairly audible ABS system, and every now and then it would randomly buzz for a second. I investigated, found the glob of iron dust, cleaned it out, and no more random ABS events. No new sensor needed. :)
(Score: 5, Informative) by c0lo on Saturday March 30 2019, @09:07PM (27 children)
Software Won’t Fix Boeing’s ‘Faulty’ Airframe [eetimes.com]
Read the linked PDF [google.com], the arguments sound pretty compelling to me.
https://www.youtube.com/watch?v=aoFiw2jMy-0 https://soylentnews.org/~MichaelDavidCrawford
(Score: 0) by Anonymous Coward on Saturday March 30 2019, @09:54PM (10 children)
Note that Mr. Travis is not an aircraft designer.
If in fact the revised engine location tends to create a pitch-up moment in some flight conditions, there are many small compensating changes possible to the airframe -- slightly larger or longer span rear horizontal tail is one obvious one. Another would be stretching the rear of the fuselage slightly (behind the main wing) to give the tail surfaces more leverage. Until all the facts are in, I'm not holding my breath because I think it's quite possible that one or more small changes were made during the design of this 737 variant.
I think Travis has a bone to pick with Boeing, anyone have any ideas why he might be mad at the company?
(Score: 5, Informative) by c0lo on Saturday March 30 2019, @10:13PM (9 children)
Note that neither are you.
Also note that Mr Travis owns a Cesna and have quite a big number of hours of real flight and flying commercial simulators.
From the linked:
---
Some hundred of deaths isn't a bone enough for you?
https://www.youtube.com/watch?v=aoFiw2jMy-0 https://soylentnews.org/~MichaelDavidCrawford
(Score: 0, Interesting) by Anonymous Coward on Saturday March 30 2019, @11:28PM (6 children)
It is obvious that he doesn't know what he is talking about in regards to the 737 and I'm almost positive has never been a commercial pilot. I can play
First is the implication that the 737 is a fly-by-wire airplane, which it isn't. The 737 autopilot has to "press on the same things that I have to press on." This is especially true when talking about the MCAS, which turns the same trim wheels as the pilots do, using the same motors for fast adjustment that the autopilot and actual pilots use.
Second is that the 737 absolutely has a checklist for runaway trim, including the memory items, and that checklist disables the MCAS system. Diagnosing a problem with the MCAS isn't really the pilots job, but it is mentioned in the checklist for some airlines anyway.
Third is that there was training for the new aircraft that did mention the MCAS. People are complaining that it wasn't a point of emphasis, in the memory items or explicitly disable on all versions of the checklist, but the memory items already disabled the MCAS. Plus, it is easier for everyone to just follow the same 737 memory checklist for all variants, rather than introducing new ones just for this one variation.
As to the "battle of wills thing," the elevators cannot overpower the horizontal stabilizer, but that is true on most planes, including most jet aircraft and ALL 737s. However, a human can overpower the computer adjusting the trim wheels and they always had the option of turning it off.
(Score: 0) by Anonymous Coward on Sunday March 31 2019, @02:05AM
Boeing 737's had a bit of a edge condition (design flaw) with its rudder actuators in the 80's as well. It was a hydraulic double-action actuator, IIRC. Anyways, at a certain set of conditions and inputs, instead of continuing to push the rudder the direction it'd been pushing it, it'd suddenly kick back the opposite direction. The usual result of these edge cases were a few rather mysterious 737 crashes.
Boeing did quietly introduce a couple of changes to the system at some point, all the while professing there wasn't a problem in the system, but the ensuing FAA directives to make the changes at least made the news in the Seattle area back then...
(Score: 3, Insightful) by Runaway1956 on Sunday March 31 2019, @02:26AM (4 children)
Since philosophy has become a part of this discussion - that is a major error in philosophy. The pilot/captain/commander/operator/driver of any vehicle is ultimately responsible for EVERYTHING that happens aboard his craft. This has been true since the first floaty thingy was hacked out of a tree trunk, and pushed out on the water. That philosophy predates the wheel.
So, yes, it really is the pilot's job to diagnose problems with MCAS, and every other system found on his aircraft.
(Score: 0) by Anonymous Coward on Sunday March 31 2019, @03:35AM (2 children)
So you are driving your truck and all of a sudden one of the brakes engages and locks solid or the thermostat readout spikes or the engine starts having ignition failures. Are you really suggesting that you would sit there and diagnose the problem? I'd hope not, you'd make sure you got control of your rig to the best of your ability and then stop when it is safe in a safe way. It is the mechanics job to actually diagnose and fix the problem.
Similarly, the pilot isn't going to sit there and figure out why the trim motors decided to trim on their own, he is going to get control and then land when it is safe in as safe a way as possible. It is the mechanics job to actually diagnose and fix the problem. If we were talking past each other, fine; if not, please inform me how a pilot should handle such a situation and how they can diagnose the difference between an AOA sensor malfunction and a short in the trim motor wiring.
(Score: 2) by Runaway1956 on Sunday March 31 2019, @08:52AM
To the pilot, there is no difference between the AOA sensor and a short in a trim motor. But, he must diagnose that the trim system is malfunctioning and bypass it. At that point, he returns to the terminal, and reports that the trim system has been bypassed, because it was misbehaving. You're right, it's not the pilot's job to diagnose the exact reason that the system failed. It's not his job to fix the problem. But, it IS his job to report as accurately as possible what the symptoms were, to help the maintenance crew to diagnose the exact problem.
To the pilot, that is at least as obvious, as a rapidly rising pyrometer would be to me in a truck. To fix that pyrometer, all I need do is downshift, and stop standing on the accelerator. That is a common problem in mountain regions, while pulling heavy loads.
(Score: 1) by khallow on Sunday March 31 2019, @09:13PM
You might not have a choice, but to sit there and diagnose the problem. At least two of your three problems can result in truck no go.
Such as happened in the two 737 accidents?
If the trim motor responses to efforts to adjust the trim, it's not the latter.
(Score: 1, Insightful) by Anonymous Coward on Sunday March 31 2019, @09:41PM
Blaming the pilot for everything is simple but what has saved lives over the decades is treating the pilot as one component in a flow of information and control inputs.
MCAS set a trap for the pilots. Just because they could have escaped it, I don't blame them for not escaping it.
(Score: 0) by Anonymous Coward on Saturday March 30 2019, @11:33PM (1 child)
I've flown Cessna aircraft and can load up a simulation of the Concorde on my simulator. Does that make me an expert on the SR-71 Blackbird?
(Score: 1) by khallow on Sunday March 31 2019, @09:15PM
(Score: 3, Insightful) by krishnoid on Saturday March 30 2019, @10:04PM (1 child)
From TFPDF:
Finally, these sorts of disconnects are being newly revealed:
Based on this new information, management may finally start to listen!
Oh, uh, maybe not. Well, what do you expect -- it came from Emmanuel Goldstein, when 1984 was probably still fresh in everyone's minds, so it was likely glossed over then. But hey, it's 2019 now! Maybe things won't repeat themselves.
(Score: 2) by c0lo on Saturday March 30 2019, @10:17PM
ᴸᴼᴸ. OK, maybe just a chuckle.
https://www.youtube.com/watch?v=aoFiw2jMy-0 https://soylentnews.org/~MichaelDavidCrawford
(Score: -1, Troll) by Ethanol-fueled on Saturday March 30 2019, @10:14PM (6 children)
Well shit, Sherlock, then ask the pilots to pitch down. Aero-planes do have control surfaces. Are those fuckers in the cockpit even paying attention or just shooting vodka and banging stewardesses?
(Score: 2) by c0lo on Saturday March 30 2019, @10:30PM
As a pilot, you can't fight those servos and the software on 737MAX didn't ask the pilots, just overrode them. Into the ground.
Do yourself a favor, mate, take another alcohol enema and pass out for the day. Both you and S/N will be better.
https://www.youtube.com/watch?v=aoFiw2jMy-0 https://soylentnews.org/~MichaelDavidCrawford
(Score: 0) by Anonymous Coward on Saturday March 30 2019, @11:23PM (4 children)
Nope try again.
(Score: 0) by Anonymous Coward on Saturday March 30 2019, @11:38PM (3 children)
Yes, look at how this pilot is powerless to stop those trim wheels: https://www.youtube.com/watch?v=cQirIH_DuAs [youtube.com] No wait, he just grasped them with no effort.
Ad there is no way to turn it off: https://www.theairlinepilots.com/forumarchive/b737/b737memoryitems.php#i [theairlinepilots.com] No wait, it's number 3 in the memory items.
(Score: 2) by Runaway1956 on Sunday March 31 2019, @02:37AM (2 children)
The video linked to in the last discussion on this subject showed that there was indeed effort required to turn that wheel on a 737 simulator. I suspect that as conditions vary, the force required to overcome that trim wheel is going to increase. However, your second point is perfectly valid. All the information on the subject that I have been shown indicates that your item number 3 has been part of pilot training since the first 737's took to the sky. It is pretty obvious that pulling the fuse stops the problem getting any worse - then the pilot can recover relatively easily. That recovery may require varying amounts of brute force, depending on how bad things got before killing the auto systems.
(Score: 0) by Anonymous Coward on Sunday March 31 2019, @03:38AM (1 child)
I know from first-hand experience those things are a bitch to turn. My point was that they are not super difficult to stop turning, especially since they have a sort of "ratcheting" or clicking behavior.
(Score: 2) by Runaway1956 on Sunday March 31 2019, @08:54AM
OK, got ya. That did seem to be the case. The copilot stopped that wheel turning, but struggled to rotate it the other direction. When the pilot found a moment, and a free hand, the two of them rotated it with little problem. After the first few clicks on the ratchet, the copilot was able to rotate it alone, with less and less obvious effort.
(Score: 2) by RamiK on Saturday March 30 2019, @11:53PM (5 children)
Technically speaking, many stealth aircraft similarly have serious aerodynamic issues that can only be piloted via software so Boeing isn't entirely wrong to peruse a software solution instead of a frame redesign.
Quadcopters are also a good example: With software they can move with unbelievable agility. Without software, they're impossible to pilot.
compiling...
(Score: 2) by c0lo on Sunday March 31 2019, @12:07AM (1 child)
Fly-by-wire can be safe as long as the airframe is aerodynamic enough to be controlled when you lose all power [wikipedia.org]. I'm yet to see a quadcopter used for passenger transport.
https://www.youtube.com/watch?v=aoFiw2jMy-0 https://soylentnews.org/~MichaelDavidCrawford
(Score: 2) by RamiK on Sunday March 31 2019, @12:54AM
No doubt. But that's not the point. The point is that they wanted to save money by moving around parts and then compensate with software which could be done safely.
The issue is the execution. when Lockheed uses software to make something that shouldn't fly, fly, they're doing it on a huge pork budget and still fuck it up. So, when Boeing went into this with "cost savings" in mind... Well, here we are.
The Volocopter is in mass production and is scheduled for manned trials this year in Frankfurt, Dubai and Singapore.
compiling...
(Score: 0) by Anonymous Coward on Sunday March 31 2019, @02:14AM (2 children)
most modern fighter jets by design are right at the edge of dynamic self-stability, if not just past it (F-16 comes to mind, as well as the X-29 forward swept wing test plane). So they rely on fly-by-wire systems to maintain that sense of stable flying to the pilot, since the pilot is not able to make the constant sub-second adjustments to maintain stable flight, much less flying in combat situations. But with these fly-by-wire systems and lack of inherent stability, the planes are also able to react to pilot inputs much faster than they ever were able to, since they don't have to overcome the inherent stability built into previous generation fighters.
(Score: 2) by Runaway1956 on Sunday March 31 2019, @02:42AM (1 child)
So, you can fly a brick, if you have enough power available?
(Score: 0) by Anonymous Coward on Sunday March 31 2019, @03:46AM
If you have the necessary reaction time, power, and control surfaces, yes you can. All you need is the right amount of power and using the control surfaces, you can get a proper angle of attack to fly anything. The real problem is maintaining stable control over the lifting-body aircraft or missile.
(Score: 2) by DannyB on Sunday March 31 2019, @03:09PM
> Software Won’t Fix Boeing’s ‘Faulty’ Airframe
Counter example of that:
Q. How many hardware engineers does it take to change a light bulb?
A. None. We'll get the device driver team to fix it in software.
When trying to solve a problem don't ask who suffers from the problem, ask who profits from the problem.
(Score: 1, Insightful) by Anonymous Coward on Saturday March 30 2019, @09:22PM (11 children)
MCAS triggered the problem, yes. But it was still a simple runaway trim motor that is trivial to disable. It doesn't matter what caused it. American pilots though little of it when they wrote it up. There is a reason that all of the fatal airline accidents over the last few years all happen in third world countries. It's usually not the airplane's fault. It really isn't here either. I'll still go up in one, with a competent pilot that knows how the machine operates.
(Score: 0) by Anonymous Coward on Saturday March 30 2019, @09:36PM (7 children)
Nope [google.com]
(Score: 0) by Anonymous Coward on Saturday March 30 2019, @09:47PM
That isn't an article, its a pdf.
(Score: 0) by Anonymous Coward on Saturday March 30 2019, @10:06PM (5 children)
It doesn't appear that he flies 737s.
Anyway, MCAS, as badly designed as it is, only operates the electric trim. It does not push on the control column. Pulling the breaker would enable the pilot to crank the trim back to normal, and then turn around and land, because controlling the trim manually is a pain in the ass, but the airplane is still flyable. The real problem is that they don't teach these guys how to fly (it's worse in the third world). They only teach them how to work the autopilot.
(Score: 0) by Anonymous Coward on Saturday March 30 2019, @10:45PM (4 children)
FTFY.
And no, that's not the real problem. The real problem is that the airframe is inherently unstable after putting in larger engines; and the software is a departure from Boeing's past approaches and its first trip into fly-by-wire territory. A bad trip.
(Score: 2, Interesting) by Anonymous Coward on Saturday March 30 2019, @11:41PM (1 child)
This does not require additional training. The memory items on all 737s for runaway trim are identical, regardless of the variant. They are the ones who didn't follow the procedure. Doesn't matter if it was MCAS or a short in the wiring or what, THE STEPS ARE THE SAME.
(Score: 0) by Anonymous Coward on Sunday March 31 2019, @02:00AM
Here's a training video from before the -MAX was on the market, shows stab trim runaway and how they dealt with it -- https://www.youtube.com/watch?v=3pPRuFHR1co [youtube.com] The cachunka-cachunka a few minutes in are the trim wheels being powered to a nose-down trim.
(Score: 0) by Anonymous Coward on Sunday March 31 2019, @03:19AM (1 child)
The real problem is that the airframe is inherently unstable after putting in larger engines
No, the stealth fighter is inherently unstable. The 737 is not. Reduce power, let go of the controls, and the airplane will recover, including this one, once you pull the breaker and manually set the trim for best glide. Make sure you have enough altitude :-) It's real basic shit that they teach very little of these days.
I'm sorry, but you don't know what your talking about. Any pilot that does not know the book Stick and Rudder is not properly qualified to solo, much less captain an airliner.
(Score: 0) by Anonymous Coward on Sunday March 31 2019, @03:22AM
you're! goddammit!
(Score: 2) by krishnoid on Saturday March 30 2019, @10:08PM (2 children)
Can you get regulation in place that demands you be provided with that information before you get your boarding pass?
(Score: 0, Insightful) by Anonymous Coward on Sunday March 31 2019, @12:42AM (1 child)
Just avoid third world (African, Asian, Russian) airlines. Nobody else had an issue with the airplane.
(Score: 2) by DannyB on Sunday March 31 2019, @03:15PM
uh, that list soon to include US.
When trying to solve a problem don't ask who suffers from the problem, ask who profits from the problem.
(Score: 0) by Anonymous Coward on Saturday March 30 2019, @10:05PM (1 child)
Yet another article about this 737 variant and not a hint of a mention of the USA accident investigation aces -- not in TFS, TFA or in any of the comments so far.
The National Transportation Safety Board (NTSB) are the ones that should be looking at this, not the FAA and certainly not all the armchair experts. Yes, the NTSB report often takes a year to come out, this is because they use experienced people, take their time, and do a very complete investigation.
(Score: 0) by Anonymous Coward on Saturday March 30 2019, @11:06PM
What part of "National" don't you understand. These accidents took place in Ethiopia and Jihadistan, not in the USA. Our pilots know how to take off, fly and land.
(Score: 5, Interesting) by c0lo on Saturday March 30 2019, @11:48PM
Seattle Times [seattletimes.com] - "Lack of redundancies on Boeing 737 MAX system baffles some involved in developing the jet":
Quartz [qz.com] - "The word “kludge” keeps coming up when pilots and engineers discuss Boeing’s 737 Max"
Naked Capitalism [nakedcapitalism.com] - "Boeing Doubles Down on 737 Max, Rejects Need for Simulator Training"
Vox [vox.com] - "The emerging 737 MAX scandal, explained"
https://www.youtube.com/watch?v=aoFiw2jMy-0 https://soylentnews.org/~MichaelDavidCrawford
(Score: 5, Interesting) by choose another one on Saturday March 30 2019, @11:56PM (7 children)
Not a pilot either, and no longer an engineer / software dev working on stuff that flies, however I have maintained an amateur interest in the industry since those days, particularly crash investigation where software involved.
tl;dr - Boeing screwed up, painted themselves into a corner where the only way out was to bodge it and hope. Not a good idea in aviation, karma came a calling.
Long:
Starts with the 787 f*** up, loads written about that, whole books. When the first 787 rolled out it had doors made of plywood, which tells us all we need to know. At the same time Boeing is working on longer term plans for a replacement for the ancient 737 - the "nma" (New Midsize Aircraft) - well that soon stopped because they needed everyone with a clue, and all spare $$$ as well, fixing the 787 fuck up.
No worries, the kind of mid-life update to the 737, the 737-NG, is solid and selling well against the A320. Then Airbus takes a look at some of the next-gen more efficient engines, higher bypass, bigger fan, and someone has the bright idea of seeing how much fuel it would save if you stuck one (well two) under an A320 wing. The answer was about 15% (more if you stick some fancy winglets on too). So was born the 320neo.
Airlines looked at the 320neo, looked at their fuel bill, went "fuck yeah". Boeing went "erm, not good". Southwest, one of Boeing's biggest customers, went "hey airbus, nice neo". Boeing went "oh fuck".
Boeing hits full corporate panic mode (which would look real slow to software folk - aircraft dev is glacial in comparison, 737 design is from the 60s, Airbus 320 from the 80s, one of the reasons I got out of the industry was wanting to work on stuff that wasn't 20yrs old). Boeing now HAS to have a response to the neo. The nma would have been fine, nay excellent, but it's still years away due to the 787 debacle.
[Short but important digression: when building planes everything has to be certified by regulators to the rules in effect at the time, that is important becuse if you can convince everyone you are actually modifying an existing plane you only have to certify the new bits to current regulations, the older bits are "grandfathered" in under old rules. Or something.]
Boeing announces it is going to stick bigger (more efficient) engines on the 737.
Everyone goes - "er WTF, how?"*
*737 is (in)famously low to the ground (there are design goals from when it was built that no one cares about now, or indeed cared about by the time the 320 was built) - no way the engines are going to fit, longer legs would mean redesigning wing and fuselage where they fold away, so much that you probably have a new plane and thus a massive re-certification job.
Airbus is already about a metre higher off the ground - all they have to do is stick new engines on (not as simple as sounds, but way simpler than what Boeing has to do).
Boeing adds as much height to the undercarriage as it can, then shoehorns the F***BigFan half under half in front of the wing - see e.g. https://leehamnews.com/wp-content/uploads/2018/11/737NG-vs-MAX-nacelles.png [leehamnews.com]
This buggers up the pitch stability ("longitudinal stability") of the airframe at high angles of attack, NOT so that it is actually unstable (that is for fighter jets) but enough that it won't get certified - at certain points the stick force vs. angle-of-attack graph is too flat. This isn't actually due to the engine thrust line, it's due the the leading edge of the engines being far forward and creating lift, lots of). So Boeing adds a new "stability augmentation system" (there is already one on the 737 called "speed trim" to correct stick force vs. g). For full layman-readable details Bjorn's corner at Leehams has a lot of good stuff: https://leehamnews.com/category/bjorns-corner/ [leehamnews.com]
The new system is MCAS, like Speed Trim (STS) it uses the trimmable horizontal stabiliser to, when needed, effectively put the aircraft out-of-trim so the pilot has to pull or push harder, but MCAS is triggered by high AOA (it is NOT an anti-stall system though), not speed changes. This is good (remember they are in panic "go go go" mode), because all the hardware needed is already there, it's just another bit of software in the FCC.
STS is an ancient well proven system, this is just a tweaked STS using AOA, should be simple, all good so far.
Now the real problems start [NOTE this may not be the real order, just speculation, but these are all pieces of the puzzle], the real short cuts are taken and the checks and balances that were there on previous systems get taken out:
1. STS kind of trims against the pilot, but there are switches that stop it doing so when the pilot is already pulling or pushing the column far enough - if you are pulling right back, STS will NOT trim the aircraft nose down. MCAS sometimes needs to work when the stick is full back. MCAS bypasses those switches. Now you can pull back as hard as you like and the MCAS will still trim forward against you.
MCAS total authority is 0.6 degrees (I think) nose down trim, maximum, not a lot to hold against.
2. If MCAS activates and the pilot re-trims the aircraft but AOA stays high, MCAS needs to trim again (so pilot has to pull harder again). MCAS gets a "reset" function, if pilot activates the trim but AOA remains high, MCAS will activate again in 5 seconds. MCAS remembers how much trim it has added, BUT this memory is toasted with the "reset" and it starts from zero. It is assumed that if the pilot re-trims they trim the aircraft back to neutral. This assumption turns out to be false. False aviation engineering assumptions about pilot behaviour tend not to make large smoking craters in the ground.
MCAS total authority is now actually unlimited (bar the physical stops), if the pilot gets it wrong. Nobody notices this because of the pilot behaviour assumption.
3. In (according to rumour) actual flight testing, Cm-alpha is found to be worse than the modelling showed. MCAS is made more powerful to compensate.
MCAS total authority is now 2.5 degrees (although actually unlimited as above). Someone "forgets" to update the certification documents, which still show 0.6 degrees. Oops.
2.5 degrees still doesn't sound much, doesn't need that much elevator to correct for, but mess-up the re-trim once and you've got 5 degrees. Try it on a paper aeroplane - 5 degrees is a lawn dart.
4. Now it gets really (not)good. Pilots have a "type rating" to fly each type of aircraft, for 737 MAX to have same type rating as 737 NG it needs to feel exactly the same, which has constrained what Boeing can do and to some extent driven MCAS requirement (Airbus on the other hand, with full FBW, can just tweak some numbers to make the neo feel like a 320). There are certain changes that would lead to a differences course with simulator training being required, this is not cheap, airlines don't want spend $. Boeing has got away with it so far, the differences course will just be a bit of CBT on an ipad.
Southwest doesn't believe Boeing can do it without sim training, so they add a penalty clause to the purchase contract - Boeing has to pay millions per plane if sim training is required.
A 737 has two sensors for angle of attack, IF MCAS uses both sensors then if the values disagree MCAS could only turn itself off (without a third sensor to compare). IF MCAS does this (a known-failed state) it MUST (certification requirement) show an alert to the pilot that the system is unavailable (c.f "Speed Trim Fail" warning light). The pilot must have a procedure to deal with the alert. Oh, and sim training is required. Result?
Boeing made the system use only one AOA sensor so that there didn't have to be a warning light so there didn't have to be sim training so Boeing didn't have to pay Southwest millions. Allegedly.
Even the Boeing test pilots apparently did not know this was hanging of a single sensor.
5. Boeing now fucks up or fudges the FEMA (Failure modes and effects analysis) for the MCAS system in the case of erroneous activation - remember it is now driven by one sensor only with no redundancy. Possibly driven again by the no-sim-training requirement, they determine that worst case failure matches up with an existing scenario called "stab trim runaway" for which there is an existing procedure that pilots are already trained on. How awfully convenient. There is another pilot-behaviour assumption there, that pilots will recognise this new intermittent runaway behaviour (which they knew nothing about) as matching an existing procedure, and be able to do that whilst trouble shooting other failures caused by dud AOA signal (failure might not be at the sensor).
Bodge, kludge, fudge, cover-up, job done, plane flies, no plywood doors on this one. Phew, we got away with it...
(Score: 0) by Anonymous Coward on Sunday March 31 2019, @02:22AM
"No application survives first contact with its users"...
Thank you, you've provided the best background so far.
(Score: 2) by Runaway1956 on Sunday March 31 2019, @03:00AM
There have been several mentions of that single sensor. You seem to have put that into it's proper perspective. The most I could do was, "Wonder why they did that?"
(Score: 0) by Anonymous Coward on Sunday March 31 2019, @04:00AM (3 children)
Makes a lot of sense. Where is that AC who keeps saying third world pilots just have to remember how to deal with runaway trim? Meanwhile, five other warnings are going off and your plane is heading for the dirt, which is not far away. And now we have Boeing acting as if they did nothing wrong. They are making their aircraft even safer.
(Score: 0) by Anonymous Coward on Sunday March 31 2019, @04:48AM (2 children)
I'm right here. They are the pilot, they need to know the memory items, period. Now, I'm not saying Boeing did nothing wrong, but the accident wasn't Boeing's fault either; the pilots could have prevented it with the memory items. Yes, repeated trims would be disorienting, especially if the annunciator or EGPWS goes off, but any failure of the trim system such that it adjusts itself beyond the inputs provided by the pilot (including the autopilot) is, by definition, a runaway trim situation. They had multiple minutes to get it under control, and they could have silenced the annunciator while doing so. Now sure, the EGPWS is a bit more to deal with as a distraction, but there are two pilots and either one could have hit the trim cutoff switch long before that became an issue.
Plus, to point out again. There was a third situation in the news from a third-world country where following the memory items did prevent a crash. Too bad only one pilot out of five was able to remember the 737 memory items during this third-world incidents.
(Score: 5, Informative) by choose another one on Sunday March 31 2019, @08:33AM
"one pilot out of five" ???
- you can't seat five in a 737 cockpit
- there were seven pilots in total, that we know of, who have faced erroneous MCAS, six failed to cope, one flight survived only because the jumpseat pilot (the extra one) had time to go get a book out and say "maybe try this"
- read AF447 report - 30+ crews known to have faced transient UAS in cruise on A330s with iced pitots, 30+ crews it was a non-event, one crew stalled into the ocean, and THAT WASN'T GOOD ENOUGH for safety levels the industry expects
But let's look beyond that.
- Boeing made an assumption about "pilots could have prevented it with the memory items".
- Did they or the airlines test that assumption with line crews? - I mean that's the sort of thing simulators are for, right?
- But MAX doesn't need simulator training... result: there are (effectively) NO max simulators out there, you fly the max after training on NG simulator, NG simulator does not have MCAS and cannot replicate this form of stab runaway, period
And further beyond:
- MAX test flights have been tracked above Renton, flight altitude profiles look like they are MCAS testing
- the altitude loss, WITH TEST PILOTS, is allegedly 2000-3000ft - maybe the test pilots don't know the memory items?
- Ethiopian flight only reached 1000ft (AGL), lose 2000ft when you are at 1000 and you end up with exactly the Ethiopian result
(Score: 2) by janrinok on Sunday April 07 2019, @07:09AM
https://www.bbc.com/news/business-47834556 [bbc.com]
https://edition.cnn.com/2019/04/04/world/ethiopian-airlines-crash-preliminary-report-intl/index.html [cnn.com]
I can understand your belief based upon your own experiences that the crashes were due to 'pilot error' from '3rd world airlines', but it seems those of us who had more inquiring minds had already seen through the blame game and realised that something rather more serious was at fault.
The initial findings clearly identified that the pilots took all the correct actions to regain control of the aircraft and, in particular, precisely followed the newly-issued advice from Boeing. The pilots actions did not prevent the crash from occurring. The company have admitted that there is something wrong and that they are responsible for correcting it.
It is probably good that you posted as AC, as your reputation here is now somewhat tarnished. Despite your claimed experience and knowledge, you were no better informed than anyone else on this site. You weren't there, you didn't know exactly what had happened, yet you continued to wave the Boeing flag and blame anyone or anything not American. You were wrong.
(Score: 2, Interesting) by pTamok on Sunday March 31 2019, @08:57AM
You are already on a karma score of 5, so I can't vote you up, but I agree - a very good summary.
Theres a couple of crucial points you make that I don't think have been highlighted elsewhere:
I think part of the issue is human factors in identifying a trim runaway. Blip the yoke button and trim adjustment by MCAS stops. That doesn't look like a runaway. Of course, if MCAS continues to see high AoA after the timeout period, it will resume trimming, but runaway trains don't stop for a while if you touch the regulator, and cars rolling down a hill don't stop every so often so you can catch up. And, of course, you can't have an 'uncommanded trim' QR becasue STS is uncommanded trim, which is completely normal. The human factors are just one hole in the Swiss cheese lining up.
(Score: 2) by Knowledge Troll on Sunday March 31 2019, @04:47PM (2 children)
All other issues such as criminality aside I've not yet seen anyone address the sensor failures and the failure rate. The software changes are an improvement and with a redundant set of sensors driving MCAS now errors in the sensors can be detected. Because there was effectively no redundancy there are cases where a single faulty sensor lead to an aircraft being command into the ground.
We would not even know the airplanes were built with out redundancy if the sensors did not fail. We do have 2 confirmed cases of flights colliding with the ground, one of those airplanes trying to do it the day before, and reports to FAA and NASA that this has happened previously and crashes were prevented by pilot action. That's a lot of failed sensors!
The sensor failure rate seems unprecedented - what's going on? As well one of the flights that crashed looks like it had a really heavy upset immediately before the airplane started the death dive. The whole air frame shakes about. Did it jiggle a sensor loose or is this idea that a single system is responsible for the crash not correct?
Also if this investigation stops at software fixes only for MCAS I think I'll refuse to fly on a MAX. There are serious issues with oversight in the design of this thing. There is a reason the DOT and FBI are investigating the FAA.
(Score: 1) by khallow on Sunday March 31 2019, @10:09PM (1 child)
On paper, it supposedly wasn't important enough to put a second one in. With that low a priority, why should it need to be reliable? I think the sensor failure rate part of a much greater problem which coincides with your last observation.
It'll be interesting to see how Boeing was able to sell this scheme and how many heads will roll as a result.
(Score: 0) by Anonymous Coward on Monday April 01 2019, @05:40PM
My understanding* is that the 737 MAX's actually have two AOA sensors. But Boing only plugged one of them into the MCAS. I have no idea why (although a previous poster suggested that by upgrading the threat of MCAS to requiring redundancies, Boing would also have to require sim traning for pilots and then have to payback Southwest a ton of money)
*Not a pilot; have read a bunch of articles on this issue; probably understand less than half of what I've read :)
(Score: 2) by Bot on Monday April 01 2019, @12:25AM (1 child)
MCAS vs. systemd
- born to automate things that can work manually CHECK
- unable to cope with the complexity of the real world vs. its model CHECK
- overrides human input no matter the consequences CHECK
- prone to cause crashes CHECK
- impossible to disable CHECK
did somebody try grep -i lennart MCAS.c ?
Account abandoned.
(Score: 0) by Anonymous Coward on Monday April 01 2019, @05:55PM
Two comments:
You missed: - Obfuscates what is going on in the system CHECK
.
And as for, "did somebody try grep -i lennart MCAS.c ?'
Thanks! /s, I may never be able to fly again.