That's one of the major speculations! There are incidents where coin symbolism and the 'low' status of brothels were at odds. The "Brothel tokens" idea is that 'real' money was exchanged outside of the brothels (preventing 'contamination' of currency with the faces of prominent and dignified figures) in exchange for these tokens, which were used as currency inside the brothels.

The theory is just one of several since evidence is scarce on the tokens' actual usage, but it's an interesting one!