This is an annotated copy of a YouTube comment thread that I was involved in. The YouTube video was from Omar (Whole Mars Catalog) where he took a 30 minute Waymo ride in San Francisco, then repeated the journey with his Tesla using FSD.
The video is below and if you haven’t already seen it, you really should. It is a bit long, but it is a really neat demonstration of the state-of-the-art status in self-driving cars.
My Primary Comment on FSD
This comment starts thanking Omar.
Omar, first of all, thanks for all you do. This was an excellent presentation of the current state-of-the-art, even though the rush hour traffic for Tesla FSD was much more challenging than the earlier Waymo drive.
@ericelfner – me
Then I lead into some information and discussion about why Omar’s drives are so much more impressive than many others. I have come to believe that a big part of this discrepancy is the map metadata that Tesla must be using to help the FSD computer.
Second, I think map metadata is key to Tesla FSD’s ability, both today and in the future. You even discussed this with me (FSDnow) in a Twitter Space about why FSD is better for you than many of us elsewhere. In this video (48:30), you mention Waymo probably choose its route (and lanes) for easier traveling. FSD has not been optimized in this way, but could be in the future.
@ericelfner – me
From this video, there is right turn (59:56) where you mention that in past versions, it would make the right turn into the service area (side street). To me, this is a sign that Tesla has improved the map metadata and the use of that data rather than an improvement in the FSD computer software. Which is contrary to your commentary, “… a robot using just cameras.”
@ericelfner – me
Then I try to give some perspective showing how incredible Tesla’s FSD is because of its ability to be used anywhere!
From your commentary, for this drive, I would expect Waymo to have a near 100% success rate on this drive and FSD to be less than 50%. This single example was far from an equal comparison (traffic/route/etc) and again, the system’s are currently optimized differently. Perhaps Tesla could perform approximately similar to Waymo in 1 in 10 drives. But, flip this around, Waymo cannot even do 1 in a million of Tesla’s drives.
@ericelfner – me
Then, I get to the meat of my discomfort with Omar’s presentation.
At 1:03:28, you go into this discussion of “solving self driving.” This is a simplistic view and does a disservice towards progress and understanding of achieving self driving cars. Tesla consumer cars are SAE L2 now and should have some L3 in the near future. Tesla L4 cars will initially be company owned. To allow L4, Tesla will have the same challenges (logistics, support, remote driver, service, etc) that Waymo and Cruise have. Consumer owned L4 cars are a distant future.
@ericelfner – me
You come back to this “solving fsd” at 1:09:05 and 1:11:28; this is meaningless and unhelpful. Tesla FSD has had successful no intervention, no disengagement drives years ago. I agree it is absolutely amazing what Tesla has done, but we are no where near to L4 consumer owned Tesla vehicles.
@ericelfner – me
I end with some helpful time links to interesting parts of the video. It is quite long and unedited (on purpose).
Key challenges for Tesla FSD in this video
43:36 “Piece of sheet. Go around, go around, go around”
47:39 Arm gesture intervention to get around stopped truck.
53:07 “We might miss the turn. We will see.”
54:01 “Ah, come on, what are you doing?”
56:22 “What a [traffic] disaster”
Okay, perhaps I rambled and covered too many topics, but my comments were really a placeholder for me to capture some of my thoughts.
An Average Joe takes exception to my comments
The YouTube user, @avgjoe5969, disagreed with (part?) of what I was saying. His comment starts:
I Disagree. Tesla is L3 and does not intend to pass through L4 (as that allows geofencing as a crutch). They intend to transition from L3 (human override still required but aware of environment) directly to L5. From what I have seen, in this video it is not in some distant future. There is no need for Tesla to have the infrastructure that L4 requires.
@avgjoe5969
This I found to be very funny and interesting. I do NOT mean to be poking fun at this user. If fact, I almost did not post my response because it could seem mean. This is not what I was after, but he could not have more elegantly displayed my concern. [And bonus, he called himself Ave Joe.]
So, I responded with:
This is exactly what I’m talking about. The average joe, @avgjoe5969 , is coming to believe Tesla FSD will “walk on water” in the not too distant future.
@ericelfner – me
Our FSD enabled cars will give Uber rides, find non-existent addresses, disallow minors and babies without baby seats, slap riders hands when they grab for the steering wheel, return left behind cell phones, clean up puke out of the backseat, fix flat tires, deal with officers informing them of a closed road, fight false damage claims in accidents, go to court when a rider sues for “emotional damage,” …
@ericelfner – me
I was not very direct with this response. I’m unsure how well I communicated my feelings. Many people, including Omar, talk about “Solving FSD” or FSD being “Solved.” I find this to be very simplistic and problematic. There is no common definition as to what “Solving FSD” means, so people will have their own interpretation. I think for many it means that their personally owned Teslas will be about to be released into the wild as RoboTaxis to make money for the owner. I’ll call this an “Autonomous Personal RoboTaxi” as opposed to an “Autonomous Company RoboTaxi” like Waymo and Cruise.
In my comment, I was trying to drive home the fact that even if the car’s driving gets really good and can achieve SAE level-4, there is still more to be done to run a RoboTaxi.
More Watchers Join in the Discussion
Then another YouTube watcher, @carensimon4236, sides with Ave Joe. Stating:
@avgjoe5969 I totally agree with Avg Joe and Omar. Eric isnt getting it. Tesla may not have the diploma in hand, but its already L3.
@carensimon4236
Just the statement that FSD is already level-3 is disturbing. (It is level-2.) People are being SO misled and/or do not understand what is going on. He continues:
What we are seeing in the recent progression of Tesla FSD beta, is that realistically, Erics list of “challenges” aren’t really challenges at all. The basic FOUNDATION of Tesla’s self-driving (with only vision) is proving out in 11.x. The issues arent truely “challenges” anymore – it’s just training time.
What? “Training Time” So, additional neural net training time and FSD can deal with a flat tire which would be required for an “Autonomous Personal RoboTaxi.” How is this not understood?
He finishes with:
As an analogy, I’ve never summed the first 100 odd numbers before; but I as I have learned the FOUNDATION of arithmetic; it’s just setting aside time to do it. Likewise, I’m not seeing anything from Omar’s drive that is overwhelming processing power of the Tesla, requiring more instrumentation, or in anyway pushing the limits of the Tesla software.
@carensimon4236
Not worth going too deep into this, but the primary problem is that he is equating this 1 drive to being about to handle ALL self-driving situations.
My Final Words
I did go ahead and respond once more to this, just because it continues to demonstrate exactly what I’m talking about. A bit more direct this time:
@carensimon4236 I guess you are saying L3 capable, because L3 means the driver does not have to pay attention. Tesla clearly states this an ADAS system, L2. Even Omar in SF would agree. My main point is that it is a disservice to use the term “Solve FSD” because it is mostly meaningless. Also, there is some natural progression towards a self driving car. For example, a car must first be able to be extremely good L2 driver before even trying L3. although Tesla FSD is absolutely amazing, it’s still not even really a competent level two system; safety disengagements routinely happen.
@ericelfner – me
So, we can look at when it will get there. Then next we can look at, when Tesla will allow drivers to take their attention off of driving and become a level three system under certain circumstances. Then expand those allowed circumstances…
@ericelfner – me
To try to help get more quality discussion going on the progress of Tesla’s FSD, I did lay out what I see as possible milestones in the improvement of the feature. You can view those in this Tesla FSD Milestones.