Announcement

Collapse
No announcement yet.

Remote touchscreen control of iPhone via serial port - Need hardware help!

Collapse
X
  • Filter
  • Time
  • Show
Clear All
new posts

  • Remote touchscreen control of iPhone via serial port - Need hardware help!

    Problem: You can output the iPod video to a monitor in your dash but you can't control the iPod from using the touchscreen in that monitor unless you do it using a computer. I want to control my iPhone from my monitor's touchscreen without using a computer. I've done some reading on the matter but don't have the hardware smarts to know if it is possible or not. I also haven't looked at the software side yet.

    Solution: Find a way to relay touchscreen inputs to the iPod.

    Proposed approach: Using a serial touchscreen device and a TTL to RS232 converter like this or this, the raw information from the serial touchscreen should be available for processing by an iPhone application.

    [Edit: Good grief, mitchjs sells a serial cable for the iPod! This simplifies everything.]

    Either a background application which reads the data and reacts to external touchscreen inputs could be run, or perhaps a special 'front end' for iPod could be built.

    Background: After a lot of reading, I've found that the iPod/iPhone has TTL serial lines built into the connector on the bottom of the phone.
    - Here are the pinouts for the iPhone/iPod from the 3rd generation forward
    - Here is a source of parts for building a custom connector
    - Here is a thread discussing serial access on the iPhone
    - Here is a tutorial for how to access the serial port on the iPhone/iPod

    For you hardware gurus, does this look like it is possible or not?
    Originally posted by ghettocruzer
    I was gung ho on building a PC [until] just recently. However, between my new phone having internet and GPS and all...and this kit...Im starting to have trouble justfiying it haha.
    Want to:
    -Find out about the new iBug iPad install?
    -Find out about carPC's in just 5 minutes? View the Car PC 101 video

  • #2
    Hiya,

    Sadly its not that easy
    Basically, the serial lines which are on the ipod touch/iphone connector, are also used by the video cable. This is because Apple do authentication before video is sent out. Basically the Apple video cable contains an authentication chip.
    So, if you wanted video but also wanted access to the serial lines, then the only way I could see was to create some sort of 'man-in-the-middle' circuit, which would keep the authentication chip connected to the serial lines, but whenever you wanted to send a command, then to automatically disconnect these lines and connect to your TTL/RS232 device.

    I tried something like this, and made some progress, but then gave up, getting completely fed up of Apple.. I have the source code for a simple 'man in the middle' circuit using an Atmel AVR chip if you need it.
    Anyway, I also stuck some information here:
    http://360.yahoo.com/shabaz_yousaf

    It would be great if someone could pursue this further, but I was actually more interested in just getting the audio out and doing basic control. Video would be a bonus.

    Comment


    • #3
      Thanks for the information! What you said about the video makes sense as I know they put some kind of authentication in the cable.

      I think I may be Overtaken By Events (OBe'd) since Apple is enabling bluetooth on the phone. It may be simpler and more elegant to use the bluetooth in the phone to talk to a bluetooth serial controller device attached to the touchscreen.

      I've been looking on the web for bluetooth serial devices. If you know of any, post them up here!
      Originally posted by ghettocruzer
      I was gung ho on building a PC [until] just recently. However, between my new phone having internet and GPS and all...and this kit...Im starting to have trouble justfiying it haha.
      Want to:
      -Find out about the new iBug iPad install?
      -Find out about carPC's in just 5 minutes? View the Car PC 101 video

      Comment


      • #4
        Originally posted by Bugbyte View Post

        I've been looking on the web for bluetooth serial devices. If you know of any, post them up here!
        There's a few devices on this page:

        http://aaxeon.com/s.nl;jsessionid=0a...&category=8777

        the cheapest one I found is on NewEgg:

        http://www.newegg.com/Product/Produc...EAR-_-33139008
        Check out my CARPC: Project Diamond Plate

        Quick Reference:
        CPU Electrical Specs
        Power Supply Calculator

        Video Connections for newbies
        Temperature Converter
        Voltage Calculator

        Comment


        • #5
          iPhone control via onboard touchscreen computer

          Hi,
          I read your posts about controling an iPod via serial port and also having its video output on the onboard touchscreen display.
          The aproach with serial port may not be the easiest/cheapest.

          I would go for wifi since the iPod touch/iPhone already has wifi.

          Connect video out of the iPhone to the touchscreen display.

          Touchscreen inputs will be processed by the onboard computer and sent via wifi to the iPhone which runs in background a TCP server app that can inject remote touchscreen commands into the iPhone's OS.

          The onboard computer runs a TCP client app that connects to the iPhone's server..

          You will need an iPhone + video cable, onboard PC + touchscreen display + a wifi card (a cheapt USB wifi dongle).

          Easy and clean, no serial wires, no bluetooth complications (bluetooth stack with serial port support).

          Comment


          • #6
            The only problem with this is that it requires a full blown computer. This negates the reason for the iPhone.

            However, I *have* been thinking along these lines. Rather than a full blown computer, how about something like the Sheeva plug computer? It is simply a Linux server on a chip with a USB port and an ethernet port for a Wifi router.

            Then I could do what you are suggesting without having to use an expensive computer. More like an inexpensive 'hardware helper'.

            Originally posted by ghettocruzer
            I was gung ho on building a PC [until] just recently. However, between my new phone having internet and GPS and all...and this kit...Im starting to have trouble justfiying it haha.
            Want to:
            -Find out about the new iBug iPad install?
            -Find out about carPC's in just 5 minutes? View the Car PC 101 video

            Comment


            • #7
              interactive mirror

              exelent work, I am looking for for the same solution for my bathroom, My plan is to have an interactive mirror with Iphone interface. But is there possible to convert a computer into a iphone. anybody out there with such skills!



              Originally posted by Bugbyte View Post
              The only problem with this is that it requires a full blown computer. This negates the reason for the iPhone.

              However, I *have* been thinking along these lines. Rather than a full blown computer, how about something like the Sheeva plug computer? It is simply a Linux server on a chip with a USB port and an ethernet port for a Wifi router.

              Then I could do what you are suggesting without having to use an expensive computer. More like an inexpensive 'hardware helper'.

              Comment


              • #8
                I had a similar dream the other day about this. I've been searching relentlessly for information about it. I'm not super savvy by any means, but i tinker and can follow schematics pretty well. I also dream, then look and see what others have done before coming up with a solution. So that's my personal introduction....

                I think the solution to the problem is both simple and complex. Simple because sending video feed to a screen is easily done, and for $49 you can buy the cable from Apple to do this, it comes in Component and Composite types, includes audio support, and a USB plug and adapter so you could charge while you listened to your music through your car stereo and at the same time was using your GPS or what have you.

                I have found an iphone app that you could easily use to send video signal. It's called ScreenSplitr

                The touch interface is simple in concept. You'll have to correct me if I'm wrong because I have no experience with using them... but basically, it's an input device similar to a mouse or keyboard.

                Going the route where a computer is involved, there is already a program that would do exactly as we want, we just have to have a touch monitor for it. Check out Veency But, I don't want the hassle of dealing with a computer in a car with bootups and shutdowns, it's too much wasted time (and battery for us hybrid drivers).

                The other optimal way would be through Bluetooth as already suggested. I've done some poking around for information on this.
                http://www.ubiqkom.org/blog/?cat=9 I would say this is on the edge of where we are headed with a touch interface. Correct me if I'm completely wrong. Ubiq has been getting results connecting a bluetooth keyboard with his iphone. If we can maybe use similar BTnodes or whatever, maybe there's hope and it's not that far off. I think most of the work has already been done for us, we just need to combine all of these elements together.

                Comment


                • #9
                  Originally posted by hyperclimber View Post
                  I think the solution to the problem is both simple and complex. Simple because sending video feed to a screen is easily done, and for $49 you can buy the cable from Apple to do this, it comes in Component and Composite types, includes audio support, and a USB plug and adapter so you could charge while you listened to your music through your car stereo and at the same time was using your GPS or what have you.

                  I have found an iphone app that you could easily use to send video signal. It's called ScreenSplitr
                  You bet. Head over to my thread on replacing the car PC with an iPhone for some info and a video on this and other issues. You can find the link in my signature.

                  The touch interface is simple in concept. You'll have to correct me if I'm wrong because I have no experience with using them... but basically, it's an input device similar to a mouse or keyboard.

                  Going the route where a computer is involved, there is already a program that would do exactly as we want, we just have to have a touch monitor for it. Check out Veency But, I don't want the hassle of dealing with a computer in a car with bootups and shutdowns, it's too much wasted time (and battery for us hybrid drivers).

                  The other optimal way would be through Bluetooth as already suggested. I've done some poking around for information on this.
                  http://www.ubiqkom.org/blog/?cat=9 I would say this is on the edge of where we are headed with a touch interface. Correct me if I'm completely wrong. Ubiq has been getting results connecting a bluetooth keyboard with his iphone. If we can maybe use similar BTnodes or whatever, maybe there's hope and it's not that far off. I think most of the work has already been done for us, we just need to combine all of these elements together.
                  I've seen this work and it has inspired me to get a bluetooth touchscreen controller. I'm waiting for it to arrive right now.

                  I still think the solution is a simple an inexpensive web server like the Sheeva. I've found a guy who has put his OBDII data onto a web server and can access it from his iPod using php. He's using a full car PC but I think it can be done with a Linux system.

                  I think I'll stop talking about this and order the damn thing and get to work on it!
                  Originally posted by ghettocruzer
                  I was gung ho on building a PC [until] just recently. However, between my new phone having internet and GPS and all...and this kit...Im starting to have trouble justfiying it haha.
                  Want to:
                  -Find out about the new iBug iPad install?
                  -Find out about carPC's in just 5 minutes? View the Car PC 101 video

                  Comment


                  • #10
                    I've found a guy who has put his OBDII data onto a web server and can access it from his iPod using php.
                    I have a ScanguageII for this. Never drive without one! The best investment I've ever made. plus... it's got a flash drive in it so you can take the data with you (a little hack process but easy enough)

                    He's using a full car PC but I think it can be done with a Linux system.
                    My brain is still trying to wrap around the concept of why in the world would you need another linux system, or any full OS for the sake of simple I/O. The iphone has everything we need for this project, we just need to be able to send and receive the data to/from it.

                    Good luck! I'm excited to see what you come up with. I would try to get a hold of Ubiq. From what I've read, it sounds like bluetooth keyboard support is NOT available in the new iphone 3.0 release. What that means for this project I'm not quite sure....

                    Comment


                    • #11
                      BugByte, check out my post on the other thread. Essentially, I propose exactly as you do. I might suggest a GumStix, as it's smaller and really designed for this work. http://www.gumstix.com/ is the site and they have a number of these small computers. some with built in wifi, and complete power specs.

                      huh, i hadn't seen it before, but they even have an output for a small touchscreen. I think I see my project forming very quickly.

                      Comment


                      • #12
                        The other thing about bluetooth is that you get one at a time... so, if you are sending video, you are going to lose hands free.

                        Comment


                        • #13
                          raleel

                          Thanks for the GumStix link! Sounds like we could really have something going!

                          The other thing about bluetooth is that you get one at a time... so, if you are sending video, you are going to lose hands free.
                          I'm a bit confused about this statement. What I was proposing was sending video from the iPhone through the Apple AV cable, then receiving input through BlueTooth from the touch screen.

                          When I have more time I'll take a look at those GumStix. I'm getting excited for my project!

                          Comment


                          • #14
                            raleel

                            Looked some more at the gumstix. I'm liking what I see. I'm still convinced that having a linux system isn't necessary for this for a few reasons. 1) Bluetooth for sending input signals to the iPhone. 2) the iPhone is equipped with USB, Firewire, Audio and Video through the dock connector 3) iPhone is equipped with a serial port through the dock connector.

                            However, taking advantage of all of these would take some serious software hacks if bluetooth input fails. Those gumstix seem like they would be a great way to get a linux system in the mix. Seems like you know a hell of a lot more about linux than most people. I ran a few versions of linux.. oh.. about 7 years ago then switched over to mac os so I'm a bit dumb when it comes to these small linux systems.

                            The 4.3" screen they advert on the gumstix site wouldn't be much bigger than an iPhone, I'm thinking more on the lines of 7-10".

                            Here are my new concerns though if we used a small linux system... As you suggested on the other thread, using ODBII for bootup/shutdown. I'm concerned with boot/kill times. These times might have to be quicker than ODBII boot/kill times. Many years ago when I was running some of the very early versions of linux, kill switches and commands would literally kill the system and in lots of cases I would have to do some serious repair work or just a clean install because stuff would get screwed up so bad. My other concern would be that is 600mhz (the processor speed of a gumstix) a fast enough processor speed to process streaming video? What about the lag of VNC over wireless for video vs. hardwired audio? The other things I have concerns with are just due to my lack of programming skills... getting linux to boot up with no GUI (not a problem) but somehow making it establish an adhoc, VNC, and UI for VNC using a program like screensplitr

                            Geez... all this sounds like I might as well put a mac mini in the car if we're going this way... Maybe I could strip down OSX and put in on the gumstix...

                            Comment


                            • #15
                              Originally posted by hyperclimber View Post
                              raleel

                              Looked some more at the gumstix. I'm liking what I see. I'm still convinced that having a linux system isn't necessary for this for a few reasons. 1) Bluetooth for sending input signals to the iPhone. 2) the iPhone is equipped with USB, Firewire, Audio and Video through the dock connector 3) iPhone is equipped with a serial port through the dock connector.
                              1) If you use bluetooth, you lose it for other things. That is to say, if you are listening to the stereo over the forthcoming A2DP, you won't be able to use it to transmit the touch signals. Same with anything else... like you won't be able to answer the phone while touching the screen if you are using handsfree. Thus, use wifi. This is the same problem that the devtoaster guys who made Rev ran into. They chose wifi to keep the hands free available.

                              I agree that you have all of those there. You probably need an app to read the screen position of your fingers. That would go on the iphone and listen to one of the ports and translate that to the local interface. I would probably use firewire though, because it's not really being used for anything (music is over usb, mp3 tags are over serial).

                              Originally posted by hyperclimber View Post
                              However, taking advantage of all of these would take some serious software hacks if bluetooth input fails. Those gumstix seem like they would be a great way to get a linux system in the mix. Seems like you know a hell of a lot more about linux than most people. I ran a few versions of linux.. oh.. about 7 years ago then switched over to mac os so I'm a bit dumb when it comes to these small linux systems.
                              Linux admin for 10 years. Embedded systems too. Did a wifi sniffing network using something similar.

                              Originally posted by hyperclimber View Post
                              The 4.3" screen they advert on the gumstix site wouldn't be much bigger than an iPhone, I'm thinking more on the lines of 7-10".
                              sadly, this is true. It's small. It's a little bigger than the iphone display, and slightly smaller than the iphone itself.

                              Originally posted by hyperclimber View Post
                              Here are my new concerns though if we used a small linux system... As you suggested on the other thread, using ODBII for bootup/shutdown. I'm concerned with boot/kill times. These times might have to be quicker than ODBII boot/kill times. Many years ago when I was running some of the very early versions of linux, kill switches and commands would literally kill the system and in lots of cases I would have to do some serious repair work or just a clean install because stuff would get screwed up so bad.
                              this would not be an issue, as it would load the entire thing into ram and anything you have on disk would actually be read only. This is how they do it on those little routers many folks have in their houses.

                              Originally posted by hyperclimber View Post
                              My other concern would be that is 600mhz (the processor speed of a gumstix) a fast enough processor speed to process streaming video?
                              on this end, yes on the server side, it can be dicier

                              Originally posted by hyperclimber View Post
                              What about the lag of VNC over wireless for video vs. hardwired audio?
                              should be next to none for audio. Video, as I note in the other thread, is actually sorta sucky at points. For some things it's good, but if you have an actively refreshing display, it can go down hill. Like, you are not going to want to watch a movie on it. Or a rapidly refreshing graph. However, for maps and music, it's actually just fine.

                              Originally posted by hyperclimber View Post
                              The other things I have concerns with are just due to my lack of programming skills... getting linux to boot up with no GUI (not a problem) but somehow making it establish an adhoc, VNC, and UI for VNC using a program like screensplitr
                              nod, that's a problem. I'm trying to talk myself into the $500 to make this all go ($160 for a wifi dongle, $220 for the gumstix, $60 for the expansion, $50 for Rev, and a power supply)

                              Originally posted by hyperclimber View Post
                              Geez... all this sounds like I might as well put a mac mini in the car if we're going this way... Maybe I could strip down OSX and put in on the gumstix...
                              hah, not likely, but an appletv can take OS X.

                              Comment

                              Working...
                              X