Remote touchscreen control of iPhone via serial port - Need hardware help!

**Bugbyte** · 03-16-2009, 10:57 PM

Problem: You can output the iPod video to a monitor in your dash but you can't control the iPod from using the touchscreen in that monitor unless you do it using a computer. I want to control my iPhone from my monitor's touchscreen without using a computer. I've done some reading on the matter but don't have the hardware smarts to know if it is possible or not. I also haven't looked at the software side yet.

Solution: Find a way to relay touchscreen inputs to the iPod.

Proposed approach: Using a serial touchscreen device and a TTL to RS232 converter like this or this, the raw information from the serial touchscreen should be available for processing by an iPhone application.

[Edit: Good grief, mitchjs sells a serial cable for the iPod! This simplifies everything.]

Either a background application which reads the data and reacts to external touchscreen inputs could be run, or perhaps a special 'front end' for iPod could be built.

Background: After a lot of reading, I've found that the iPod/iPhone has TTL serial lines built into the connector on the bottom of the phone.
- Here are the pinouts for the iPhone/iPod from the 3rd generation forward
- Here is a source of parts for building a custom connector
- Here is a thread discussing serial access on the iPhone
- Here is a tutorial for how to access the serial port on the iPhone/iPod

For you hardware gurus, does this look like it is possible or not?

**Shabaz** · 03-17-2009, 07:50 PM

Hiya,

Sadly its not that easy

Basically, the serial lines which are on the ipod touch/iphone connector, are also used by the video cable. This is because Apple do authentication before video is sent out. Basically the Apple video cable contains an authentication chip.
So, if you wanted video but also wanted access to the serial lines, then the only way I could see was to create some sort of 'man-in-the-middle' circuit, which would keep the authentication chip connected to the serial lines, but whenever you wanted to send a command, then to automatically disconnect these lines and connect to your TTL/RS232 device.

I tried something like this, and made some progress, but then gave up, getting completely fed up of Apple.. I have the source code for a simple 'man in the middle' circuit using an Atmel AVR chip if you need it.
Anyway, I also stuck some information here:
http://360.yahoo.com/shabaz_yousaf

It would be great if someone could pursue this further, but I was actually more interested in just getting the audio out and doing basic control. Video would be a bonus.

**Bugbyte** · 03-19-2009, 12:12 PM

Thanks for the information! What you said about the video makes sense as I know they put some kind of authentication in the cable.

I think I may be Overtaken By Events (OBe'd) since Apple is enabling bluetooth on the phone. It may be simpler and more elegant to use the bluetooth in the phone to talk to a bluetooth serial controller device attached to the touchscreen.

I've been looking on the web for bluetooth serial devices. If you know of any, post them up here!

**dMand** · 03-19-2009, 02:52 PM

Originally Posted by Bugbyte

I've been looking on the web for bluetooth serial devices. If you know of any, post them up here!

There's a few devices on this page:

http://aaxeon.com/s.nl;jsessionid=0a...&category=8777

the cheapest one I found is on NewEgg:

http://www.newegg.com/Product/Produc...EAR-_-33139008

**gigel** · 03-27-2009, 12:50 PM

Hi,
I read your posts about controling an iPod via serial port and also having its video output on the onboard touchscreen display.
The aproach with serial port may not be the easiest/cheapest.

I would go for wifi since the iPod touch/iPhone already has wifi.

Connect video out of the iPhone to the touchscreen display.

Touchscreen inputs will be processed by the onboard computer and sent via wifi to the iPhone which runs in background a TCP server app that can inject remote touchscreen commands into the iPhone's OS.

The onboard computer runs a TCP client app that connects to the iPhone's server..

You will need an iPhone + video cable, onboard PC + touchscreen display + a wifi card (a cheapt USB wifi dongle).

Easy and clean, no serial wires, no bluetooth complications (bluetooth stack with serial port support).

**Bugbyte** · 03-30-2009, 04:24 PM

The only problem with this is that it requires a full blown computer. This negates the reason for the iPhone.

However, I *have* been thinking along these lines. Rather than a full blown computer, how about something like the Sheeva plug computer? It is simply a Linux server on a chip with a USB port and an ethernet port for a Wifi router.

Then I could do what you are suggesting without having to use an expensive computer. More like an inexpensive 'hardware helper'.

**crxmaniac** · 04-03-2009, 09:20 PM

exelent work, I am looking for for the same solution for my bathroom, My plan is to have an interactive mirror with Iphone interface. But is there possible to convert a computer into a iphone. anybody out there with such skills!

Originally Posted by Bugbyte

The only problem with this is that it requires a full blown computer. This negates the reason for the iPhone.

However, I *have* been thinking along these lines. Rather than a full blown computer, how about something like the Sheeva plug computer? It is simply a Linux server on a chip with a USB port and an ethernet port for a Wifi router.

Then I could do what you are suggesting without having to use an expensive computer. More like an inexpensive 'hardware helper'.

**hyperclimber** · 04-06-2009, 10:59 PM

I had a similar dream the other day about this. I've been searching relentlessly for information about it. I'm not super savvy by any means, but i tinker and can follow schematics pretty well. I also dream, then look and see what others have done before coming up with a solution. So that's my personal introduction....

I think the solution to the problem is both simple and complex. Simple because sending video feed to a screen is easily done, and for $49 you can buy the cable from Apple to do this, it comes in Component and Composite types, includes audio support, and a USB plug and adapter so you could charge while you listened to your music through your car stereo and at the same time was using your GPS or what have you.

I have found an iphone app that you could easily use to send video signal. It's called ScreenSplitr

The touch interface is simple in concept. You'll have to correct me if I'm wrong because I have no experience with using them... but basically, it's an input device similar to a mouse or keyboard.

Going the route where a computer is involved, there is already a program that would do exactly as we want, we just have to have a touch monitor for it. Check out Veency But, I don't want the hassle of dealing with a computer in a car with bootups and shutdowns, it's too much wasted time (and battery for us hybrid drivers).

The other optimal way would be through Bluetooth as already suggested. I've done some poking around for information on this.
http://www.ubiqkom.org/blog/?cat=9 I would say this is on the edge of where we are headed with a touch interface. Correct me if I'm completely wrong. Ubiq has been getting results connecting a bluetooth keyboard with his iphone. If we can maybe use similar BTnodes or whatever, maybe there's hope and it's not that far off. I think most of the work has already been done for us, we just need to combine all of these elements together.

**Bugbyte** · 04-07-2009, 09:30 AM

Originally Posted by hyperclimber

I think the solution to the problem is both simple and complex. Simple because sending video feed to a screen is easily done, and for $49 you can buy the cable from Apple to do this, it comes in Component and Composite types, includes audio support, and a USB plug and adapter so you could charge while you listened to your music through your car stereo and at the same time was using your GPS or what have you.

I have found an iphone app that you could easily use to send video signal. It's called ScreenSplitr

You bet. Head over to my thread on replacing the car PC with an iPhone for some info and a video on this and other issues. You can find the link in my signature.

The touch interface is simple in concept. You'll have to correct me if I'm wrong because I have no experience with using them... but basically, it's an input device similar to a mouse or keyboard.

Going the route where a computer is involved, there is already a program that would do exactly as we want, we just have to have a touch monitor for it. Check out Veency But, I don't want the hassle of dealing with a computer in a car with bootups and shutdowns, it's too much wasted time (and battery for us hybrid drivers).

The other optimal way would be through Bluetooth as already suggested. I've done some poking around for information on this.
http://www.ubiqkom.org/blog/?cat=9 I would say this is on the edge of where we are headed with a touch interface. Correct me if I'm completely wrong. Ubiq has been getting results connecting a bluetooth keyboard with his iphone. If we can maybe use similar BTnodes or whatever, maybe there's hope and it's not that far off. I think most of the work has already been done for us, we just need to combine all of these elements together.

I've seen this work and it has inspired me to get a bluetooth touchscreen controller. I'm waiting for it to arrive right now.

I still think the solution is a simple an inexpensive web server like the Sheeva. I've found a guy who has put his OBDII data onto a web server and can access it from his iPod using php. He's using a full car PC but I think it can be done with a Linux system.

I think I'll stop talking about this and order the damn thing and get to work on it!

**hyperclimber** · 04-07-2009, 03:13 PM

I've found a guy who has put his OBDII data onto a web server and can access it from his iPod using php.

I have a ScanguageII for this. Never drive without one! The best investment I've ever made. plus... it's got a flash drive in it so you can take the data with you (a little hack process but easy enough)

He's using a full car PC but I think it can be done with a Linux system.

My brain is still trying to wrap around the concept of why in the world would you need another linux system, or any full OS for the sake of simple I/O. The iphone has everything we need for this project, we just need to be able to send and receive the data to/from it.

Good luck! I'm excited to see what you come up with. I would try to get a hold of Ubiq. From what I've read, it sounds like bluetooth keyboard support is NOT available in the new iphone 3.0 release. What that means for this project I'm not quite sure....