It's true. But then most tourists manage to avoid the less scenic parts of The City.
Inland districts within blocks of City Hall, like the Tenderloin and Western Edition. SF can be a peculiar combination of extreme wealth and poverty separated only by short distances. Some districts like Potrero Hill have evolved with gentrification, driving out the poorer people to the East Bay. The Mission District and the Panhandle could be a little dangerous at night, but nicer in the daytime. I'd probably say the same about the Haight as well.
Though I haven't been in SF in years, and have no desire to go much further back into California than Truckee or South Lake Tahoe these days. Having to wade through a gauntlet of panhandlers as soon as I exited the Montgomery St. BART station every day. Yeah, commuting to and working in the financial district for ten years was more than enough for me.
Nothing I haven't experienced in a number of other major US cities. With SF as no real exception in this regard.
San Francisco's downtown area is more contaminated with drug needles, garbage, and feces than some of the world's poorest slums