If you were to go outside and see a bunch of millennials mindlessly milling around the street – outside of your home even – swiping their fingers across their smart phones while they hold it out at arm’s distance, then chances are they’re playing Pokémon Go. Most people don’t mind this recent cultural phenomenon which has players roaming around the block in hopes of capturing Pokémon.

But not everyone is too happy about the new Pokemon game. Some people have been complaining about the locations players go to play their game. Many places, including churches and government institutions, have seen an influx of people. This is because these places have either been marked as a PokeStop or a Gym for trainers, and many of these places want The Pokémon Company to give them ability to opt out of using their location for Pokémon Go. According to Gamespot, J.C. Smith of The Pokémon Company has stated that they are working on a new update that will have features that are

appealing to the fans but also respectful of the private institutions that are affected by it.

In the few weeks since it’s launched, Pokémon Go has already been removed from cultural and historical institutions like the Holocaust Museum in Washington D.C. However, places like the Hiroshima Peace Memorial Park and the Arlington National Cemetery are still requesting to be removed. One man in Massachusetts wants his home removed as a Gym due to all the people passing by everyday:

It’s been a lot of fun, but I think it’s a good idea to try and get the gym moved. It turns out there’s a great park right across the street from my house. A simple fix may be nudging the coordinates a few degrees and then everyone’s happier. The neighborhood keeps the gym, and I get a little more privacy.

Pokemon Go players water bombed in Rhodes by residents. Waiting for revenge of the nerds pic.twitter.com/QF1nQD9Tb3

— Brianna Parkins (@parkinsbrea) July 13, 2016

According to Kotaku, residents of a suburban city in Sydney Australia have become upset with the sheer amount of people gathering outside their apartments. One resident, who took the picture above, claimed that there were hundreds of Pokémon Go trainers outside until 3:00 AM, and it got so bad that they had to call police to the area to move them out. Perhaps players are getting a bit carried away with Pokémon Go and maybe the new update will keep institutions like memorials and places of worship free of PokeStops and Gyms. Until then, people will continue to go where their Pokédex guides them.