'Pokémon Go' players unknowingly trained delivery robots with 30B images

I like Pokemon Go and play almost every day. I did this scan one time and then stopped. The rewards are not worth the hassle. I don't think many players are doing it. It's just very weird to stand somewhere and scan an object.

I also wouldn't say 'unknowingly trained', it's pretty obvious what it does, and I think the game even tells you that they want to understands how the POI looks like in 3D.

Niantic are a number of people who are doing this. Its not that clear from the article, but niantic spatial are using the images captured from users to create a 3d model of "THE WORLD" or where people play pokemon go.

They have then fed that data into a more modern version of colmap (https://github.com/colmap/colmap) to create a point cloud. Then the engineering to make sure that point cloud is aligned accurately and automatically.

Once you have that point cloud aligned to the world, all you need is another image with some overlapping feature. Using simple trigonometry you can work out where the camera is from one picture

This is largely trivial to do for a few 100 sqaure meters. the hard part is doing it fast in at the city scale. Extracting a few thousand features from an image and then matching them against >billion other points is hard to do quickly, without some optimisations.

The thing that is not mentioned here is that data freshness is actually more important. Building change (advertising hoardings, paint jobs, logo changes, building remodelled etc) so the data goes stale. Its actually not that expensive anymore to just send your own people to scan areas. (A number of startups pre 2020 did it, mapillary provides a platform for it, although now owned by facebook)

The robots will be feeding that data back in to the map. the special sauce is updating the map without infringing patents, and doing it efficiently.

Niantic has been doing this for a long time starting with Ingress. I've maybe done a handful of scans in PoGo but as others have mentioned the rewards were just not worth the trouble. The rewards for doing it in Ingress were much better (at least back in the day).

I'm more split on my feelings towards it these days given our current political/social climate but part of me still thinks the idea of mapping the real world in great detail is a worthy endeavor if it can be done "right". I'd probably be more inclined to support it if they would release the data or make it publicly accessible for others to use but it being tied to the whims of a corporation (even one that's been less shitty than most) makes it hard to get behind.

At what point will we have people transmit their car dash cams along with GPS information in order to generate more data? I’m actually surprised this hasn’t happened yet with self driving car manufacturers needing more and more data

Despite the lack of transparency, is this so bad? Players are being given a game in exchange for collectively building a database.

> crowdsourced data, seemingly collected for one purpose

> Whether players knew it or not, those scans were creating 3D models of the real world

Kind of shitty reporting. Did users know about this data collection or not? Was it not disclosed?

And we’re all training ai right now with our comments. Even the bots here.

Not unexpected, but it looks the oldest kid, Ingress, is being ignored again. IIRC, there was some badge you could earn by doing a number of those scans.

Or is Ingress even still around?

Does anyone remember Ingress? I always wondered what it was we were training by playing that game.

Millions of software devs unknowingly trained LLMs to start replacing their jobs.

PUSH your code to fit to find out more.

Looks like teenagers are going to have fun playing Pokemon Go, and now have faster food deliveries.

It's useful to map the world, this is what Google / Baidu / Yandex Maps are doing too.

Big corporations have found the way to make us work for free in their own terms. The balance of power between the working class and capital is totally broken.

And for me it is not just the lack of transparency. It is the power balance. I should not need to work for free, give my data, and god knows what to play a game. I should not be living knowing that I am being exploited at each interaction with software. Transparency is good, but not enough. "Click here to accept" and thousands of lines of legalese do not create a fair society.

wasnt this sort of obvious to anyone familiar with Niantic?

Nintendo's brand means nothing.

I think that only the author of this post didn't know that. Everybody know that Niantic is a big data company.

Pizza delivery robot, or a live grenade delivery robot, depending on the country and the dataset buyer.

I don't think this is the worst thing trained.

Niantic builds massive geospatial models that can localize and reconstruct views: https://www.nianticspatial.com/

Extremely detailed mappings of CONUS with spatial intelligence already built around it, and we let the company get sold to Saudi government last year.

Pretty sure you take a video when you’re doing this task so the images are extra extracted from the video?

I'm not positive this was a secret (See: Reddit post about it from 2018):

https://www.reddit.com/r/TheSilphRoad/comments/8i7byi/pokemo...

Stalking as a service - upload a photo of your target and get back gps coordinates, coming soon!

I don't think anyone actually cares about this in principle. This is more of a product and marketing problem than a legal or moral one.

What people dislike is noticing the strings attached so distastefully. I can't think of any fads or pastimes where there aren't any, but the benefits of the activity offered should outweigh the cost.

In that sense, Pokémon Go was a bad deal. I still don't get what was ever in it for the player.

Those pokestop scans are trash. I can't believe they will get enough detailed information out of them to allow navigation by robots.

I knew, or strongly assumed, and think it’s a great dynamic.

> The massive crowdsourcing effort could use real-world to help robots deliver pizza.

Huh? Does popsci not have copy editors?

[dead]

My friend plays Pokémon Go for hours every day while walking his dog. I asked him about this and now we’re both confused. The in game scanning is only for major landmarks in the game. Even in his dense city these landmarks are few and far between. The world model would only have sparse information in the area immediately surrounding these landmarks.

I don’t know if there’s much substance to the delivery robot story. This could be a journalist trying to make the story relatable.