Changing KML graph options

longfeltwant · 06-09-2009, 03:32 AM

I am working on new useful visualizations for the generated KML. My Jeep does not have MAF, so I cannot use the visualization of Speed, MPG, and Position, which would probably be the first most obvious option.

First though I modified the code to allow me to pass in two colors for the "high" color and the "low" color. (In heightandcolor.c

Code:

unsigned int deltaColor = (highcolor - lowcolor) / numcols;

		for(i=1;i<=numcols;i++) {
			fprintf(f, "<Style id=\"%s%i\">\n"
				"<PolyStyle>\n"
				"<color>ff%06x</color>\n"
				"</PolyStyle>\n"
				"<LineStyle>\n"
				"<color>ff%06x</color>\n"
				"</LineStyle>\n"
				"</Style>\n", styleprefix, i,
				highcolor - (deltaColor * (i-1)),
				highcolor - (deltaColor * (i-1))
			);
		}

I kept the original kmlvalueheightcolor function signature by passing red and green to the modified-and-renamed function, so that the rest of the obd2kml code would still run.

Because I can't use Speed, MPG, and Position, I implemented Speed, RPM, and Position. It is cool because it shows how when you rev the engine, you gain speed. (This code is in obdgpskml.c; coloredheightkml is the name of the improved kmlvalueheightcolor.)

Code:

		coloredheightkml(db,f,graphname, "",
			"vss",maxaltitude, "rpm", 5, 0xFF0000, 0x00FF00, 1,
			sqlite3_column_double(trip_stmt, 1), sqlite3_column_double(trip_stmt, 2));

Let me say here that the color coding for KML objects is four two-digit hex codes, for a total of eight digits, zero thru F, where the first two mean BLACK, the second two BLUE, the third two GREEN, the last two RED. It took me forever to figure that out.

That all works. The graph output is pretty cool. The next one I want to try is fuel themed: Fuel tank level (height), fuel trim (color), position. That will show how the fuel tank is drained over time. I changed the codes collected and logged so I can play with different metrics:

Code:

static struct obdservicecmd obdcmds[] = {
	{ 0x00, 4, NULL,            "PIDs supported 00-20" , 0, 0, "Bit Encoded", NULL },
	{ 0x01, 4, NULL,            "Monitor status since DTCs cleared" , 0, 0, "Bit Encoded", NULL },
	{ 0x02, 4, NULL,            "DTC that caused required freeze frame data storage" , 0, 0, "Bit Encoded", NULL },
	{ 0x03, 8, NULL,            "Fuel system 1 and 2 status" , 0, 0, "Bit Encoded", NULL },
	{ 0x04, 2, NULL,            "Calculated LOAD Value" , 0, 100, "%", obdConvert_04 },
	{ 0x05, 1, "temp",          "Engine Coolant Temperature" , -40, 215, "Celsius", obdConvert_05 },
	{ 0x06, 1, NULL,            "Short Term Fuel Trim - Bank 1", -100, 99.22, "%", obdConvert_06_09 },
	{ 0x07, 1, NULL,            "Long Term Fuel Trim - Bank 1", -100, 99.22, "%", obdConvert_06_09 },
	{ 0x08, 1, NULL,            "Short Term Fuel Trim - Bank 2", -100, 99.22, "%", obdConvert_06_09 },
	{ 0x09, 1, NULL,            "Long Term Fuel Trim - Bank 2", -100, 99.22, "%", obdConvert_06_09 },
	{ 0x0A, 1, NULL,            "Fuel Rail Pressure (gauge)", -100, 99.22, "%", obdConvert_0A },
	{ 0x0B, 1, "maf",            "Intake Manifold Absolute Pressure", 0, 765, "kPa", obdConvert_0B },
	{ 0x0C, 2, "rpm",           "Engine RPM", 0, 16383.75, "kPa", obdConvert_0C },
	{ 0x0D, 1, "vss",           "Vehicle Speed Sensor", 0, 255, "km/h", obdConvert_0D },
	{ 0x0E, 1, NULL,            "Ignition Timing Advance for #1 Cylinder", -64, 63.5, "degrees relative to #1 cylinder", obdConvert_0E },
	{ 0x0F, 1, "iat",            "Intake Air Temperature", -40, 215, "Celsius", obdConvert_0F },
	{ 0x10, 2, NULL,           "Air Flow Rate from Mass Air Flow Sensor", 0, 655.35, "g/s", obdConvert_10 },
	{ 0x11, 1, "throttlepos",   "Absolute Throttle Position", 1, 100, "%", obdConvert_11 },
	{ 0x12, 1, NULL,            "Commanded Secondary Air Status" , 0, 0, "Bit Encoded", NULL },
	{ 0x13, 1, NULL,            "Location of Oxygen Sensors" , 0, 0, "Bit Encoded", NULL },
	{ 0x14, 2, NULL,            "Bank 1 - Sensor 1/Bank 1 - Sensor 1 Oxygen Sensor Output Voltage / Short Term Fuel Trim", 0, 1.275, "V", obdConvert_14_1B },
	{ 0x15, 2, NULL,            "Bank 1 - Sensor 2/Bank 1 - Sensor 2 Oxygen Sensor Output Voltage / Short Term Fuel Trim", 0, 1.275, "V", obdConvert_14_1B },
	{ 0x16, 2, NULL,            "Bank 1 - Sensor 3/Bank 2 - Sensor 1 Oxygen Sensor Output Voltage / Short Term Fuel Trim", 0, 1.275, "V", obdConvert_14_1B },
	{ 0x17, 2, NULL,            "Bank 1 - Sensor 4/Bank 2 - Sensor 2 Oxygen Sensor Output Voltage / Short Term Fuel Trim", 0, 1.275, "V", obdConvert_14_1B },
	{ 0x18, 2, NULL,            "Bank 2 - Sensor 1/Bank 3 - Sensor 1 Oxygen Sensor Output Voltage / Short Term Fuel Trim", 0, 1.275, "V", obdConvert_14_1B },
	{ 0x19, 2, NULL,            "Bank 2 - Sensor 2/Bank 3 - Sensor 2 Oxygen Sensor Output Voltage / Short Term Fuel Trim", 0, 1.275, "V", obdConvert_14_1B },
	{ 0x1A, 2, NULL,            "Bank 2 - Sensor 3/Bank 4 - Sensor 1 Oxygen Sensor Output Voltage / Short Term Fuel Trim", 0, 1.275, "V", obdConvert_14_1B },
	{ 0x1B, 2, NULL,            "Bank 2 - Sensor 4/Bank 4 - Sensor 2 Oxygen Sensor Output Voltage / Short Term Fuel Trim", 0, 1.275, "V", obdConvert_14_1B },
	{ 0x1C, 1, NULL,            "OBD requirements to which vehicle is designed" , 0, 0, "Bit Encoded", NULL },
	{ 0x1D, 1, NULL,            "Location of oxygen sensors" , 0, 0, "Bit Encoded", NULL },
	{ 0x1E, 1, NULL,            "Auxiliary Input Status" , 0, 0, "Bit Encoded", NULL },
	{ 0x1F, 2, NULL,            "Time Since Engine Start", 0, 65535, "seconds", obdConvert_1F },
	{ 0x20, 4, NULL,            "PIDs supported 21-40" , 0, 0, "Bit Encoded", NULL },
	{ 0x21, 4, NULL,            "Distance Travelled While MIL is Activated", 0, 65535, "km", obdConvert_21 },
	{ 0x22, 2, NULL,            "Fuel Rail Pressure relative to manifold vacuum", 0, 5177.265, "kPa", obdConvert_22 },
	{ 0x23, 2, NULL,            "Fuel Rail Pressure (diesel)", 0, 655350, "kPa", obdConvert_23 },
	{ 0x24, 4, NULL,            "Bank 1 - Sensor 1/Bank 1 - Sensor 1 (wide range O2S) Oxygen Sensors Equivalence Ratio (lambda) / Voltage", 0, 2, "(ratio)", obdConvert_24_2B },
	{ 0x25, 4, NULL,            "Bank 1 - Sensor 2/Bank 1 - Sensor 2 (wide range O2S) Oxygen Sensors Equivalence Ratio (lambda) / Voltage", 0, 2, "(ratio)", obdConvert_24_2B },
	{ 0x26, 4, NULL,            "Bank 1 - Sensor 3 /Bank 2 - Sensor 1(wide range O2S) Oxygen Sensors Equivalence Ratio (lambda) / Voltage", 0, 2, "(ratio)", obdConvert_24_2B },
	{ 0x27, 4, NULL,            "Bank 1 - Sensor 4 /Bank 2 - Sensor 2(wide range O2S) Oxygen Sensors Equivalence Ratio (lambda) / Voltage", 0, 2, "(ratio)", obdConvert_24_2B },
	{ 0x28, 4, NULL,            "Bank 2 - Sensor 1 /Bank 3 - Sensor 1(wide range O2S) Oxygen Sensors Equivalence Ratio (lambda) / Voltage", 0, 2, "(ratio)", obdConvert_24_2B },
	{ 0x29, 4, NULL,            "Bank 2 - Sensor 2 /Bank 3 - Sensor 2(wide range O2S) Oxygen Sensors Equivalence Ratio (lambda) / Voltage", 0, 2, "(ratio)", obdConvert_24_2B },
	{ 0x2A, 4, NULL,            "Bank 2 - Sensor 3 /Bank 4 - Sensor 1(wide range O2S) Oxygen Sensors Equivalence Ratio (lambda) / Voltage", 0, 2, "(ratio)", obdConvert_24_2B },
	{ 0x2B, 4, NULL,            "Bank 2 - Sensor 4 /Bank 4 - Sensor 2(wide range O2S) Oxygen Sensors Equivalence Ratio (lambda) / Voltage", 0, 2, "(ratio)", obdConvert_24_2B },
	{ 0x2C, 1, NULL,            "Commanded EGR", 0, 100, "%", obdConvert_2C },
	{ 0x2D, 1, NULL,            "EGR Error", -100, 99.22, "%", obdConvert_2D },
	{ 0x2E, 1, NULL,            "Commanded Evaporative Purge", 0, 100, "%", obdConvert_2E },
	{ 0x2F, 1, "fuel",            "Fuel Level Input", 0, 100, "%", obdConvert_2F },
	{ 0x30, 1, NULL,            "Number of warm-ups since diagnostic trouble codes cleared", 0, 255, "", obdConvert_30 },
	{ 0x31, 2, NULL,            "Distance since diagnostic trouble codes cleared", 0, 65535, "km", obdConvert_31 },
	{ 0x32, 2, NULL,            "Evap System Vapour Pressure", -8192, 8192, "Pa", obdConvert_32 },
	{ 0x33, 1, "baro",            "Barometric Pressure", 0, 255, "kPa", obdConvert_33 },
	{ 0x34, 4, NULL,            "Bank 1 - Sensor 1/Bank 1 - Sensor 1 (wide range O2S) Oxygen Sensors Equivalence Ratio (lambda) / Current", 0, 2, "(ratio)", obdConvert_34_3B },
	{ 0x35, 4, NULL,            "Bank 1 - Sensor 2/Bank 1 - Sensor 2 (wide range O2S) Oxygen Sensors Equivalence Ratio (lambda) / Current", 0, 2, "(ratio)", obdConvert_34_3B },
	{ 0x36, 4, NULL,            "Bank 1 - Sensor 3/Bank 2 - Sensor 1 (wide range O2S) Oxygen Sensors Equivalence Ratio (lambda) / Current", 0, 2, "(ratio)", obdConvert_34_3B },
	{ 0x37, 4, NULL,            "Bank 1 - Sensor 4/Bank 2 - Sensor 2 (wide range O2S) Oxygen Sensors Equivalence Ratio (lambda) / Current", 0, 2, "(ratio)", obdConvert_34_3B },
	{ 0x38, 4, NULL,            "Bank 2 - Sensor 1/Bank 3 - Sensor 1 (wide range O2S) Oxygen Sensors Equivalence Ratio (lambda) / Current", 0, 2, "(ratio)", obdConvert_34_3B },
	{ 0x39, 4, NULL,            "Bank 2 - Sensor 2/Bank 3 - Sensor 2 (wide range O2S) Oxygen Sensors Equivalence Ratio (lambda) / Current", 0, 2, "(ratio)", obdConvert_34_3B },
	{ 0x3A, 4, NULL,            "Bank 2 - Sensor 3/Bank 4 - Sensor 1 (wide range O2S) Oxygen Sensors Equivalence Ratio (lambda) / Current", 0, 2, "(ratio)", obdConvert_34_3B },
	{ 0x3B, 4, NULL,            "Bank 2 - Sensor 4/Bank 4 - Sensor 2 (wide range O2S) Oxygen Sensors Equivalence Ratio (lambda) / Current", 0, 2, "(ratio)", obdConvert_34_3B },
	{ 0x3C, 2, NULL,            "Catalyst Temperature Bank 1 /  Sensor 1", -40, 6513.5, "Celsius", obdConvert_3C_3F },
	{ 0x3D, 2, NULL,            "Catalyst Temperature Bank 2 /  Sensor 1", -40, 6513.5, "Celsius", obdConvert_3C_3F },
	{ 0x3E, 2, NULL,            "Catalyst Temperature Bank 1 /  Sensor 2", -40, 6513.5, "Celsius", obdConvert_3C_3F },
	{ 0x3F, 2, NULL,            "Catalyst Temperature Bank 2 /  Sensor 2", -40, 6513.5, "Celsius", obdConvert_3C_3F },
	{ 0x40, 4, NULL,            "PIDs supported 41-60" , 0, 0, "Bit Encoded", NULL },
	{ 0x41, 4, NULL,            "Monitor status this driving cycle" , 0, 0, "Bit Encoded", NULL },
	{ 0x42, 2, NULL,            "Control module voltage", 0, 65535, "V", obdConvert_42 },
	{ 0x43, 2, NULL,            "Absolute Load Value", 0, 25700, "%", obdConvert_43 },
	{ 0x44, 2, NULL,            "Commanded Equivalence Ratio", 0, 2, "(ratio)", obdConvert_44 },
	{ 0x45, 1, NULL,            "Relative Throttle Position", 0, 100, "%", obdConvert_45 },
	{ 0x46, 1, NULL,            "Ambient air temperature", -40, 215, "Celsius", obdConvert_46 },
	{ 0x47, 1, NULL,            "Absolute Throttle Position B", 0, 100, "%", obdConvert_47_4B },
	{ 0x48, 1, NULL,            "Absolute Throttle Position C", 0, 100, "%", obdConvert_47_4B },
	{ 0x49, 1, NULL,            "Accelerator Pedal Position D", 0, 100, "%", obdConvert_47_4B },
	{ 0x4A, 1, NULL,            "Accelerator Pedal Position E", 0, 100, "%", obdConvert_47_4B },
	{ 0x4B, 1, NULL,            "Accelerator Pedal Position F", 0, 100, "%", obdConvert_47_4B },
	{ 0x4C, 1, NULL,            "Commanded Throttle Actuator Control", 0, 100, "%", obdConvert_4C },
	{ 0x4D, 2, NULL,            "Time run by the engine while MIL activated", 0, 65525, "minutes", obdConvert_4D },
	{ 0x4E, 2, NULL,            "Time since diagnostic trouble codes cleared", 0, 65535, "minutes", obdConvert_4E },
	{ 0x51, 2, NULL,            "Fuel Type", 0, 0, "", NULL },
	{ 0x52, 2, NULL,            "Ethanol fuel %", 0, 100, "%", obdConvert_52 },
	{ 0x00, 0, NULL,            NULL }
};

Does anyone have any other cool ideas? In addition to my desire to link in photographs, I also want a utility that will add an entry to a "note" table in the database, along with a timestamp. Then the KML output could include these notes along the way, as placemarks. Also, if we were to monitor code 0x01, could we collect DTCs in real time? If so, we could also stash those in the notes table, so that the output could show DTC information. I'll play with that next time I have a chance.

chunkyks · 06-09-2009, 12:13 PM

Awesome stuff!

Post patches and lemme know if you'd like to be attributed /a la/ "name <email>", "name <url>" or something else... :-)

Gary (-;

longfeltwant · 06-09-2009, 01:48 PM

Thanks a lot, chunk. Can you tell me how to submit a patch? Would I post entire code files, or would I use the 'patch' utility? (I have never used the 'patch' utility, although I understand conceptually what it does.)

chunkyks · 06-09-2009, 02:54 PM

Short version:

Code:

svn diff > ../patch-for-something.diff

Long version: If your patch does multiple things, it's better to send individual patches for each thing. For example, so long as your kml changes don't impact anything else, it'd be preferable to send me a diff for that, and then a separate one for other obdinfo changes. So you'd do something like this:

Code:

cd top-level-svn-checkout
svn update
svn diff src/kml > ../kml-color-and-new-chart.diff
svn diff src/obdinfo > ../obdinfo-add-fuel-columns.diff

And then mail me those two patches. Common good practice is to send them to the mailing list, unless they're extremely large, in which case you post a link on the mailing list or just mail it straight to me. As a whole, you'll find people are hesitant to accept extremely large patches in general, though.

You'll never find two developers who do stuff the same way, this is just what works for me. I'm vanishingly unlikely to turn down modifications just because you sent a patch inline instead of attached, or directly to me instead of the ml, or any of these things.

If you're interested, the other half of the process looks like this:

I download the patch into a directory, then use

Code:

patch -p0 < something.diff
# ... or, if it's been compressed,
# gzip -cd something.diff.gz | patch -p0

-p0 says "don't strip any of the path from the diff". If you look at the diff itself [it's a very readable plaintext file], you'll see that it has path details in it. For example:

Code:

Index: src/logger/main.h
===================================================================
--- src/logger/main.h   (revision 105)
+++ src/logger/main.h   (working copy)
@@ -37,6 +37,7 @@
        { "no-autotrip", no_argument, NULL, 'n' }, ///< Disable automatic trip starting and stopping
        { "spam-stdout", no_argument, NULL, 't' }, ///< Spam readings to stdout
        { "serial-log", required_argument, NULL, 'l' }, ///< Log serial port data transfer
+       { "braiiiinns", no_argument, NULL, 'b' }, ///< Tastes like braiiiinnns
        { "enable-optimisations", no_argument, NULL, 'o' }, ///< Enable elm optimisations
        { "db", required_argument, NULL, 'd' }, ///< Database file
        { NULL, 0, NULL, 0 } ///< End

I could apply that from the top-level directory with -p0, or I could go into the logger directory and apply it with -p2 [which would tell it to strip the two leading directories from the path to the files]

Anyways. Looking around for good tutorials on patch etiquette, I didn't find much coherent stuff except this. It's written by one of the more incendiary people in the open source world and is a bit unnecessarily aggressive, but it's mostly still accurate. [for example, since we're using svn you can ignore the stuff about diff formats; the default is fine]

My apologies if you already knew this stuff, I was just looking for a decent resource for someone who had to ask how to create patches :-)

Gary (-;

longfeltwant · 06-12-2009, 01:25 PM

(Hi. I'll work on that patch soon.)

Although I tried to enable logging for the options I listed above, only PIDs under 0x20 are saved to the database. I notice that PID 0x20 gets a list of supported commands above #20, which led me to this function:

void *getobdcapabilities(int obd_serial_port)

I did my best to understand that code, and I think it is only reporting capability for PIDs returned by command 0x00, which gets a list of supported commands between 0x00 and 0x20. Can you confirm or deny that void getobdcapabilities does not capture PIDs over 0x20? If so, is there a reason not to try to get PIDs greater than 0x20?

longfeltwant · 06-12-2009, 01:44 PM

I went ahead and made the patch, which I have attached (changed .diff to .txt).

I did this so you could see the work I'm doing, but I don't encourage you to fold all the changes into your code. Specifically, in obdgpskml.c, I commented out all your visualizations in favor of the one I wrote -- and I imagine you wouldn't want that.

The tiny good improvement I made was to the kmlvalueheightcolor function. It still exists with the same function signature, but now just passes thru to my new coloredheightkml function, which differs because it takes two hex colors as parameters. That code, you might want to accept, now or later. Oh, except my function name stinks and should be improved.

chunkyks · 06-12-2009, 02:15 PM

I love short and long versions. Dunno why, but there you go:

Short version:
If printobdcapabilities [ie, "obdgpslogger -p"] shows everything [including the ones >=0x20], then isobdcapabilitysupported *should* return the right value. Can you empirically try checking this?

Run "obdgpslogger -p", pick one above 0x20, and call isobdcapabilitysupported with that pid, see what happens. If it returns 1, then you know the problem isn't there.

If you know your car supports something above 20, but obdgpslogger -p doesn't report it, then that's the problem right there.

Long version:
getobdcapabilities works thus [I'll pull bits of code and paste them in to show you]:

1) 0x00, 0x20, 0x40, etc, are all "capabilities" pids. They tell you which pids are supported up to the next one. It's not in the spec now, but in future, I'm sure that 0x80 will do the same thing.

2) So I iterate across those capabilities pids [this is the part you actually asked about]:

Code:

while(true) {

   /// stuff goes here

  if(D&0x01) { // If PID 0x20 after the one we're currently looking at is supported...
    current_cmd += 0x20; // Go to that next one and loop again
  } else {
    break;   // Otherwise we're done
  }
}

3) So at the top of the loop, we grab the fields and turn them into a big ol' bit pattern [according to the C spec, "long" is guaranteed to be *at least* 32 bits [although it is usually imlemented as 64 on modern processors]. "int" is guaranteed to be *at least* 16 bits [although it usually implemented as 32 on modern processors - but we cannot safely rely on it]]:

Code:

cap_status = getobdbytes(obd_serial_port, current_cmd, 0,
                   &A, &B, &C, &D, &bytes_returned);

// error checking .....

// Make those four bytes into a bit pattern
unsigned long val;
val = (unsigned long)A*(256*256*256) + (unsigned long)B*(256*256) + (unsigned long)C*(256) + (unsigned long)D;

4) Now to actually find out what the car thinks it's capable of. We have a bit pattern, so we have to iterate across all those bits and find out if they're set or not:

Code:

for(c=current_cmd+1, currbit=31 ; currbit>=0 ; currbit--, c++) {
          // iterate [currbit] across all the bits 31..0
          //  and store a counter [c] that represents the PID for that bit
  if(val & ((unsigned long)1<<currbit)) {
        // Check the bit in our bit pattern. If it's set, remember that
    struct obdcapabilities *nextcap = (struct obdcapabilities *)malloc(sizeof(struct obdcapabilities));
    nextcap->next = NULL;
    nextcap->pid = c;
    curr_cap->next = nextcap;
    curr_cap = nextcap;
  }
}

It's possible the problem lies somewhere else, of course. But that's the logic thus far.

Gary (-;

chunkyks · 06-12-2009, 02:18 PM

Quote:

Specifically, in obdgpskml.c, I commented out all your visualizations in favor of the one I wrote -- and I imagine you wouldn't want that

Heh. that's why I put in the defaultvisibility parameter. You can make just the one you want visible as default, and leave the rest turned off - they're there in google earth, you just can't see them until you check the checkbox.

Thanks for the patch! I'll look at it later, kinda busy coding something else at work right now :-(

Gary (-;