While it is possible to permanently disable VSP, I don't think anyone here is advocating that.
My hybrid doesn't have a sound effect generator, and there have been times I've had to toot the horn at sighted people ambling down the middle of a parking aisle, oblivious to the car creeping behind them. Something like a VSP would have been great to have instead of the blare of the horn. I also got into the habit of tapping the horn before backing out of a spot. Driving slowly through a residential neighborhood, it is always possible for a child (or even a smartphone-obsessed adult these days) to pop out into the street from in-between parked cars.
The 2011 US-spec LEAF came with a temporary override button to silence the VSP, say for not disturbing the neighbors when arriving home late at night. The switch was removed for 2012+ US market cars, likely in anticipation of US Pedestrian Safety Enhancement Act of 2010. The compliance deadline had since been delayed from 2014 to 2018.
Good background info here:https://en.wikipedia.org/wiki/Electric_vehicle_warning_sounds
VSP as it is now is an imperfect solution. It is sometimes too loud (late night) or not loud enough (busy parking lot). The backup bell is intermittent and hard to identify where or what it is coming from. The forward motion whoosh does not play during backing maneuvers. It would make more sense to have a backup sound emit from under the rear bumper. Some are unhappy that VSP doesn't have an "idling" sound. An improved VSP could continuously monitor the ambient environmental sounds and sculpt the frequencies and amplitudes as appropriate, sort of like auto-brightness on your phone.
The addition of driver-assistance technologies such as cameras and distance sensors, automatic braking, etc, should prove to be a valuable complement to things like VSP. Rear-view and all-around cameras are incredibly helpful already.
I'm patiently awaiting the day when all cars are much quieter. As a pedestrian, you almost need hearing protection at five- and seven-lane intersections with all the idling engines, let alone the exhaust fumes and heat.