There's a lotta variables to this.(´～`)
Are the individuals that know me simply told that I frequent this website with no context? Then no harm is done because Heyuri means nothing to even most internet users. (unless they have prior knowledge about the website.)
Are the people in my local vicinity, not only made aware of the name but are also shown the overboard as an example of the content that appears on this site? They're gonna see some weird stuff but not really a consistent amount of anything.
Now the mysterious informant could also say that this is a lolicon website, but wouldn't that just be bad framing?
Is this really a lolicon website? /b/ is the most popular board and it's not the only image board to have loli content, 8chan.moe has a lolicon board. It happens to have a dedicated board.
We could set up a poll but I don't think everyone that comes here is a lolicon.
This is a place that would anger internet prudes and lawful individuals (maybe a few other parties) but most people won't understand this place nor its culture.