How well can you map? (archive)

rweait's picture

A treasure map drawn on a rough bit of wall with stylish X

What limits the quality of your contributions to OpenStreetMap data? The answer is surprisingly simple.

I propose a simple Three-Star system to categorize your edits according to how much value you personally add to the data you contribute to OpenStreetMap.

Three-star edits

three stars

Three-star edits occur in places you know like the back of your hand, places that you visit every two weeks or more often. You notice the subtle things. Nothing significant can get past you for more than two weeks. This part of the map is always within days of being up to date. Anything that lasts less than two weeks may well be too fleeting to consider editing in OpenStreetMap anyway.

Three-star edits are in your home territory, and probably include your home, the places where you walk the dog, your weekly shopping locations, your office and most frequent customer locations. The route from home to work does not count as part of your three-star territory unless you leave early enough that you can stop and map. No mapping without stopping the car, bus or bike. That wouldn't be safe.

Two-star edits

two stars

Two-star edits occur in places that you know really well. You visit your two-star territory every two months, perhaps every three months at the outside. You're really familiar with the place and the context, but things can get by without you noticing. When did the bakery change hands? And that shoe shop; wasn't that the dry cleaners? You keep the big stuff up to date every time you visit because you would notice the new traffic signals or bicycle lane.

Your two-star territory might include the neighbourhood of friends and relatives who you see several times each year, the shopping district or favourite restaurant across town.

One-star edits

one star

If you see a place every two or three years it counts as part of your one-star territory. That hotel at SotM last year? Those are one-star edits, at best; it has been more than three months ago, unless you have been back to Girona since then. You understand a little of the local context but has the road alignment changed since last July? You can't possibly know.

Zero-star edits

Everything else is a zero-star edit. We hope that our contribution is as good as the source we use. And we try to make reasonable choices when de-conflicting multiple sources. Even zero-star edits can add some value to OpenStreetMap, but that value isn't personal. We are drawn to map places that interest us, either places we've been long ago, or places we've read about in the news. The opportunity to map from old memories must be tempered by awareness that old data should not replace new data. If a local mapper has added your school and it disagrees with your memory; the local mapper may well have mapped changes that occurred since you left.

Okay, I was in Washington, DC last month...

... so everything I edit in DC is a two-star edit.

Well, no. You have to actually be there. Not just near there. To count yourself as present for the Three-Star system you have to be on the ground (I totally flew over the Grand Canyon before I landed in Las Vegas; One-star edits for all of Arizona!), with line of sight, or hands-on access to survey a site. Otherwise you can't count it for your edits. Being asleep on the bus when it drove through Flagstaff doesn't count either.

Surely there are modifiers for this simple system?

Why yes, there are modifiers. The Three-Star system serves not as a floor for the minimum quality of your edits, but as a ceiling. No clever use of tags can add a half-star to an edit but errors can remove stars.

Your edits degrade over time. What was a three-star edit last month is only a two-star edit this month, unless you have returned to confirm your survey. Your two-star edits degrade to one-star after a few months. After a year or two your old one-star edits become zero-star edits. Common sources might even be more recent than your last survey there.

See the benefit to OpenStreetMap? If you are always there, OSM is always up to date.

If you make an error in tagging, connectivity, spelling or other error, remove half of the stars for that edit, then drop any fractional star.

k: amenity, v: calf
k: name, v: Coffee Time

Add this amenity in your home range and your nominally three-star edit becomes a one-star. Fortunately, these errors are easily corrected on your next visit within a few days.

If you aren't there, you can't do any better than common sources. A dozen other mappers from around the world could use the same three year-old aerial imagery, and year-old vector data and come up with a dozen variants of what you mapped without a substantial difference. You can't do anything to distinguish your contribution if you haven't been there.

The greatest benefit that you bring to OpenStreetMap is your local data. You know where the public sources are wrong and you've fixed them. You know when things change, and you update them before the next publication cycle of the authoritative source. You can observe things that are not included in other sources and provide that data to OpenStreetMap.

This is preposterous!

Yes. It's silly and fun and nobody should take the Three-Star rating system seriously. But it is correct. Take away local knowledge and mappers are almost completely interchangeable. What's left to distinguish mappers and data quality without local knowledge?

Quality of the mapper

But I'm a better mapper than that other mapper!

Yes, of course. You are a beautiful and unique snowflake of a mapper. We are all beautiful and unique mappers and we are all better than the others.

Mappers learn from other mappers and get better over time. So this is self-correcting. We can eliminate mapper quality as a contributor to quality of an edit.

Quality of the source material

Our sources are lying to us. You know it's true. We might as well say it out loud. All of our sources lie.

We know that our GPSes lie to us. We can take the same route, every day for weeks and the traces never align perfectly. They'll cluster for sure, but they are never identical. The GPS always lies. A better GPS that lies at a lower magnitude breaks our hearts all the more when it inevitably pops out a bogus result.

Imagery lies to us as well. Rectification can be off, stitching can leave artefacts and angular offset can hide interesting objects behind others. Yuck.

Source data from which we may import has drawbacks as well. It was collected from sources that lie as well. No getting around that. Just because a file comes from an authoritative source does not make it perfect. It can't be. At best, source files have well-understood limits on their errors. But they'll often just flat out have mistakes too. You've seen it dozens of times.

We can compare and combine sources. What do you do with three sources that disagree in different ways? Split the difference? Take a wild guess? Pick your favourite source? How about choose the data that two of the three sources agree upon?

Sorry. Just kidding. I know that sources never agree unless the two different sources are actually one file with two different names.

We can't do a better job of mapping than our source material does. We can smooth curves, but we're just guessing.

But our sources are lying to all of us at the same time. With no discrimination, any mapper can come up with the same result from the same incorrect sources. No mapper can make perfect results from imperfect data.

So data quality does not contribute to the quality of an edit either.

A mapper on every block

That's what we want. Tell your friends.

This simple Three-Star system ignores social effects. There is great benefit to the project in introducing new mappers to OSM. How do you compare the benefit of one-star mapping for an hour to introducing a new contributor to OSM for an hour?

Simple. You save that for a future article.

This archived article was published on 02/21/2011 - 04:51


Pirate map photo by eleanor ryan is CC-By-SA