Page 6 of 7 FirstFirst 1234567 LastLast
Results 51 to 60 of 70

Thread: Crowdsourced Maps Project - mods help/thoughts

  1. #51
    Organic Me
    Auto Apps:loading...
    Heather's Avatar
    Join Date
    Sep 2007
    Location
    Fells Point, Baltimore, MD
    Posts
    1,287
    Blog Entries
    48
    I have been in contact with OSM for almost 6 months now, starting with an OSM meet up I attended in Baltimore. They have been going through some staffing changes and slow to respond. This is why they did not attend the AFK Fest which had originally been intended as an mp3Car-OSM event.

    I have had multiple emails with the founder of OSM letting him know that we seek to help him. In particular we have asked for information from them regarding 1. what data they would like and 2. how they would like it. So far the only thing that we know is that mp3Car will most likely need to be able to verify that the data is real before we dump it on them. But i do know that we can do a data dump, as they have done this in the past.

    So - i think that while we wait for feedback, let's start uploading for two reason -1. to show OSM that we intent to provide data that will help them and 2. to have deta to show them, which may allow them to provide better feedback in terms of what data they would like and how they would like it.

    I emailed the founder last week with a link to this forum and lambosprit's application. I will wait another few days for a response and then follow up with OSM again. Sean will be working with lambospirit this afternoon to test out the new version and document setup, and Goce is documenting how you can access the data via ftp.

    FYI - if you are wondering why mp3Car is pushing this, I can explain. I have a personal interest in helping put a dent in map data fees and navigation costs. As I have seen the invoices first hand, I am aware that these costs and upfront license requirements inhibit companies like Centrafuse from being able to create an excellent product. Agreed no need to reinvent the wheel, this data should help all others out there like OSM. However, I also think that innovation comes from where you least expect, and if forum members are interested in actually creating the maps and collaborating on routing engines...mp3Car will try to help in whatever way possible including using contacts that we have to bring in external resources. We do not see that this will drive profit, but rather grow the community which is something that we are firmly committed to doing.

  2. #52
    Super Moderator
    Auto Apps:loading...
    JohnWPB's Avatar
    Join Date
    Sep 2005
    Location
    West Palm Beach, Florida
    Posts
    4,723
    Blog Entries
    1
    I would like to see a template/example of the data for each point to be collected.

    I can put together a script that will be included with the upcoming release of the DFX skin, that can collect that data. When the skin is run for the very first time, a pop-up will explain what it is, and they can opt in or out of collecting the data.

    I will have the plugin require that a 4 satellite GPS lock is available, to help keep the data as accurate as possible. If it is less than 3 satellites, no log entry will be made.

    I will then need to create a script that they can run periodically that will upload the data to a server ect.

    I know this is a RR solution only, but being it has the largest number of users in the forums, is a good way to start gathering the data for sure.

    I just need to know all the detail before I can start to put this together.
    Front End of Choice: Ride Runner (Is there anything else??? ) & Powered by the DFX5.1 Skin Available in the Mobile App Mart

    My Fiero Build Thread

  3. #53
    Car Audio Moderator durwood's Avatar
    Join Date
    Sep 2006
    Location
    Chicagoland (St. Charles,IL)
    Posts
    2,028
    Quote Originally Posted by justchat_1 View Post
    there we go...now everyones on the same page.

    As far as data storage and interoperability - whos in contact with the OSM developers? Can we find out if .GPX files are easiest for them or if they would prefer batch adding via an API? Also, can we batch add under a single username (ex: mp3car) or will they want unique usernames?
    http://wiki.openstreetmap.org/wiki/Upload
    http://wiki.openstreetmap.org/wiki/Api06#GPS_Traces

  4. #54
    Raw Wave
    Auto Apps:loading...
    justchat_1's Avatar
    Join Date
    Jul 2008
    Location
    Boston, Ma or NY,NY
    Posts
    1,783
    If you scrolled up-you would see thats not a viable solution:
    http://wiki.openstreetmap.org/wiki/A...Authentication

    Bugbyte & Heather:
    Good info...will wait to hear back on that.

    Some suggestions for the first pass (to hopefully take the best first step we can);
    For the xml submitted:
    * Change sequential numbering to a relative timestamp (makes it alot easier to validate-potentially useful for other things down the road)
    * Add hdop and vdop to the submitted data (again-for error filtering)

    For the serverside stuff:
    * Use http not ftp for submission -> it deals with packet loss a lot better. Allow a raw xml POST, or a zipped file upload.
    * On upload everything should be dumped to an sql database for storage
    * A separate thread then processes all of the data in the queue...checking data validity and filtering as necessary to ensure an accurate submission...and finally packing it up and dumping it to OSM

    Other things to consider:
    * Many users want to ensure there is no way they can be tracked using this system, and be completely anonymous
    * At the same time we need to ensure the data is valid and coming from a reliable source
    * Would some type of one way unique id be beneficial in ensuring source validity? Possibly a salted hash?
    * Would an ip based blacklist be enough?

    Thoughts?

  5. #55
    Organic Me
    Auto Apps:loading...
    Heather's Avatar
    Join Date
    Sep 2007
    Location
    Fells Point, Baltimore, MD
    Posts
    1,287
    Blog Entries
    48
    Quote Originally Posted by justchat_1 View Post

    For the serverside stuff:
    * Use http not ftp for submission -> it deals with packet loss a lot better. Allow a raw xml POST, or a zipped file upload.
    * On upload everything should be dumped to an sql database for storage
    * A separate thread then processes all of the data in the queue...checking data validity and filtering as necessary to ensure an accurate submission...and finally packing it up and dumping it to OSM
    Ok - let me talk to Goce

    Quote Originally Posted by justchat_1 View Post

    Other things to consider:
    * Many users want to ensure there is no way they can be tracked using this system, and be completely anonymous
    * At the same time we need to ensure the data is valid and coming from a reliable source
    * Would some type of one way unique id be beneficial in ensuring source validity? Possibly a salted hash?
    * Would an ip based blacklist be enough?

    Thoughts?
    I thought that we are generating a unique identifier....so that we can weed out bad data providers. We discussed this and thought that banning an IP is really not sufficient.
    Am i wrong?

    In terms of privacy - The unique identifier should only be known by the uploader - they should hang onto it... mp3Car will not maintain a database of unique identifier and name so that we do not know who is driving where. I am unsure if this is what is happening based on the description of the app.
    can you comment lambosprit?

    Also - don't we have the program cutting off the tracks at some random place between x and y miles from where the tracks began. Lambosprit explained here: Crowdsourced Map Data Collection App - How it works. in sticky above.

  6. #56
    Maximum Bitrate
    Auto Apps:loading...
    lambosprit's Avatar
    Join Date
    Sep 2006
    Location
    UK
    Posts
    786
    Quote Originally Posted by Heather View Post
    Ok - let me talk to Goce
    If this is the way you want it done just tell me and I'll amend my app.


    Quote Originally Posted by Heather View Post
    I thought that we are generating a unique identifier....so that we can weed out bad data providers. We discussed this and thought that banning an IP is really not sufficient.
    Am i wrong?
    Nope. the app generates its own unique id when its installed. Its only known to the user so provides anonymity. However if people want to see how many miles, points etc they've logged then they could search using it or reveal it if they wish. Its up to us to decide how we want to show that.

    Quote Originally Posted by Heather View Post
    In terms of privacy - The unique identifier should only be known by the uploader - they should hang onto it... mp3Car will not maintain a database of unique identifier and name so that we do not know who is driving where. I am unsure if this is what is happening based on the description of the app.
    can you comment lambosprit?
    Thats exactly whats happening.

    Quote Originally Posted by Heather View Post
    Also - don't we have the program cutting off the tracks at some random place between x and y miles from where the tracks began. Lambosprit explained here: Crowdsourced Map Data Collection App - How it works. in sticky above.
    yes. again for privacy we delete a random number of points (with a user definable minimum) from the beginning and end of each track. And yes I did explain that in the sticky.

  7. #57
    Raw Wave
    Auto Apps:loading...
    justchat_1's Avatar
    Join Date
    Jul 2008
    Location
    Boston, Ma or NY,NY
    Posts
    1,783
    Quote Originally Posted by lambosprit View Post
    Nope. the app generates its own unique id when its installed. Its only known to the user so provides anonymity. However if people want to see how many miles, points etc they've logged then they could search using it or reveal it if they wish. Its up to us to decide how we want to show that.
    Quote Originally Posted by lambosprit View Post
    In terms of privacy - The unique identifier should only be known by the uploader - they should hang onto it... mp3Car will not maintain a database of unique identifier and name so that we do not know who is driving where. I am unsure if this is what is happening based on the description of the app.
    can you comment lambosprit?
    Thats exactly whats happening.
    In terms of real world privacy that really doesn't offer any. This still allows anyone with your "uniqueid" access to everywhere you've been. It also could potentially be subpoenaed and used in court. Are either of those cases likely? probably not-but I do remember hearing that being a concern from some forums members.
    More importantly for me, is the complete lack of data protection a client side uniqueid provides. Its a sad but true fact that you put a system with potential to be used for malicious purposes and someone will try. Especially given the open specs, someone could easily add 100,000 bogus tracks to the database in a few minutes...and then do the same thing again the next day.
    Quote Originally Posted by lambosprit View Post
    yes. again for privacy we delete a random number of points (with a user definable minimum) from the beginning and end of each track. And yes I did explain that in the sticky.
    Well according to that sticky:
    It ignores a random number of points when the app is started or comes out of hibernation/suspend.
    It deletes a random number of collected points when the app is started or comes out of hibernate/suspend (this is because most of my tracks where to and from work so every end of journey showed where I started previously, not good for privacy!)
    I was just saying if there will be other implementations of this idea it would help if that was more specific (i was thinking it was some fixed range). So your saying its a user selectable range of points removed from the start and end of each trip?

  8. #58
    Maximum Bitrate
    Auto Apps:loading...
    lambosprit's Avatar
    Join Date
    Sep 2006
    Location
    UK
    Posts
    786
    Quote Originally Posted by justchat_1 View Post
    I was just saying if there will be other implementations of this idea it would help if that was more specific (i was thinking it was some fixed range). So your saying its a user selectable range of points removed from the start and end of each trip?
    No its random but with a min setting. so the user can say that atleast 10 points are ignore but it could be more. This was done because of suggestions from another user about privacy, as was the client side user id.

    My opinion is that we have to get people on board and to do that they need to be sure of their privacy. At the moment nobodies uploading let alone putting junk in.

  9. #59
    Organic Me
    Auto Apps:loading...
    Heather's Avatar
    Join Date
    Sep 2007
    Location
    Fells Point, Baltimore, MD
    Posts
    1,287
    Blog Entries
    48

    Feedback from OpenStreetMap on this project

    On 2 Oct 2009, at 08:05, Heather Sarkissian wrote:
    > Steve
    >
    > I would like to have the mp3Car forums help OSM by providing data to
    > improve your existing maps.

    Cool!

    > The mp3Car forum is full of car computer users from around the world
    > who can easily collect NMEA and other data and automatically submit
    > it to us. We are working with our forum members to automate the
    > process of collecting the data from car computers. We will next
    > move on to collection from iPhones.
    >
    > Could you let me know:
    > 1. If you can use NMEA data to help generate the maps and

    It's part of the problem - we use it as an input but at best it gives
    you geometry - no metadata.

    > 2. What other data might be useful?

    PoIs, streetnames, oneway information...

    > There is an explanation of the collection application that one of
    > our forum members created to automate collection: http://www.mp3car.com/vbulletin/crow...how-works.html
    >
    > Feel free to post in the forums or you can email me any thoughts
    > that you would have in terms of what data OSM needs, how OSM would
    > like to receive it, and whether our current application will provide
    > this data.

    So that app you link to is basically there, it just needs to spit out
    GPX files instead of a bespoke XML format.

    Yours &c.

    Steve

  10. #60
    Raw Wave
    Auto Apps:loading...
    justchat_1's Avatar
    Join Date
    Jul 2008
    Location
    Boston, Ma or NY,NY
    Posts
    1,783
    Also, can we batch add under a single username (ex: mp3car) or will they want unique usernames?
    ^ We still need to know that ^

    Also, if you can ask what type of safeguards they want on data integrity? Do we just need to check for device related errors? prevent flooding? or do we need to be able to provide accountability? (ex: if someone submits bad data prevent them from doing it again)

Page 6 of 7 FirstFirst 1234567 LastLast

Similar Threads

  1. Crowdsourced Map Data Collection, Mapping and Routing: project explained.
    By Heather in forum Crowdsourced Data Collection, Open Source Mapping And Routing
    Replies: 0
    Last Post: 10-02-2009, 09:26 AM
  2. Ambitious Pico-ITX-based Car PC Project
    By Kenshin5 in forum Newbie
    Replies: 12
    Last Post: 12-18-2007, 05:14 PM
  3. Project UA6, 1 week old, some new mods
    By lookinco in forum Show off your project
    Replies: 17
    Last Post: 06-26-2006, 12:52 AM
  4. A Start Community Project
    By BenjaminS in forum MacCar
    Replies: 9
    Last Post: 03-25-2005, 06:46 PM

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •