TimLee wrote:But the poorly named Tesla system left a technology savy driver lulled into the perception it was autonomous which it is NOT and a behaviour pattern that resulted in his torso being sheared in half going under a tractor trailer.
This was truly a tragedy, and my prayers are with his family. However, I don't think the name of the system was the key issue that led to that accident. Rather, it appears that, from his experience using it
, Joshua Brown became overly confident in the robustness of the AutoPilot system. Had he been using an identically-named though less polished system in a different brand of automobile, I doubt he would have developed such overconfidence. Perhaps the most important lesson in all of this is, as others have pointed out, that human beings are often overly quick to trust automated systems. To counter this tendency, I'm guessing Tesla will be forced to put more "nagging" in the system.
TimLee wrote:Why the mainstream manufacturers that are more risk adverse than risk taking Tesla are doing two orders of magnitude more testing BEFORE providing such systems.
That may be true, but I'm not convinced that lack of testing was the issue here. After all, as of the time of the fatality, Tesla customers had accumulated roughly 100 million miles with AutoPilot enabled. The real challenge, I think, is that the current AutoPilot system is simply not capable of fully autonomous driving under any real-world conditions, nor could it be with the current sensors and computing hardware. For what it is, it's quite impressive, better than competing systems. Hence the overconfidence in it.
Tesla will undoubtedly keep making AutoPilot better, including addressing the shortcomings that led to Joshua Brown's untimely death. Some, including myself, are speculating that Tesla will release V2 of the AutoPilot hardware soon (if NHTSA/NTSB don't nix that). But even if AutoPilot succeeds in greatly reducing accidents and fatalities compared to human-only driving, the media and the public will probably continue to be very exacting when accidents do occur.