I grew up in San Jose, and still pass through downtown quite often, here are some tips:
- There's a crosswalk at the lightrail station in front of the convention, you can use that to get across the street. If you need to go further without encountering the people at or near the Market/San Carlos intersection (the people you're describing tend to cluster around downtown all year long), I recommend cutting through the business plaza behind the Tech, or hugging the Fairmont/Art Museum side of the downtown park
- Travelling in groups won't entirely deter agressive panhandlers and catcallers, but it'll supply a buffer between you and them, as well as making them think twice. You can ask a group of friends/fellow cosplayers to go with you - people inside the convention tend to be a friendly lot, and wouldn't mind offering their support.
- When you do encounter persistant vendors and/or people who set off alarms of any sort, keep walking. Don't stop to engage them in any way, and do not acknowledge them in any way.
- Pretty much all areas east and south of the convention center should probably be avoided - neighborhood quality takes a quick dive going south, and between us and the University, it's not as friendly as it could be.
Beyond that, I've got nothing. San Jose's always been this way growing up, and in recent years it's looking more like LA-lite. Right now, the best that can probably be done is to report all problems surrounding your stay with FanimeCon, and ask for solutions.