It’s the name with the explicit connection to James, a leader of the early church.
There’s a difference between the idea of a pseudo-fictional composite character, like King Arthur who was constructed centuries after his time, and a real historical guy who existed and had stories written about what he said.
Consider how much evidence we have for Pythagoras. Pythagoras was also a weird religious cult leader, but I’d expect most here would know him for the Pythagorean theorem. Which he didn’t come up with. Does that not make him enough of a “Pythagoras” for you?
You have to gauge your sense of skepticism. There’s a difference between “oh, Gilgamesh seems to be showing up in all of these King’s List documents that claim thousands of years of dynastic dominance which are 80% bullshit to oil up a kingship’s perceived position in the world.” and “oh, here’s a bunch of texts about an unusual rogue ‘rabbi’ that developed a following; there’s some probably exaggerated claims of healing, an oddly novel resurrection story that has more added to it as each Gospel is written.”
Read just the resurrection, Mark-> Matthew-> Luke->John to see how the more fanciful stuff develops. Heck - maybe even read the New Testament in chronological order - starting with the letters of Paul and see them as dealing with situations happening in real time. Treat it as a ‘found footage’/‘ambiguous narrator’ collection. A murder mystery.
There’s a difference between reading the Bible as a religious text, to either prove or disprove, and as a compilation of vastly different documents, by vastly different authors, writing across centuries.
For a modern example, think about John of God - one of the faith healing charlatans that Oprah promoted. Will people who live in the next few centuries automatically discount his existence because they find it occasionally next to a description of his supposed miracles, which accounts are perhaps more likely to survive than those of his sexual assault allegations? Will the things that he will have said to have said not be accurate, even if other information about him is not?
At the end of the day, there’s just as much evidence for the existence of several early historical figures that we don’t doubt the existence of. I think it’s reasonable to not privilege the text as anything other than a primary source document, and recognize that a lot of similar supernatural claims have been added to multiple real world figures in history.