As you have stated, a DVM isn't the most reliable way to detect an oscilating change. An inductor will attempt to "fight" any change in potential across it. As a result of this simplified fundamental, as the input voltage drops, the regulator is attempting to still maintain the output voltage, and as a result it is working the inductor harder and harder, causing a spike on the output of the design will occur. The best way to measure such a spike and determine the possible effects on any equipment connected is with a storage oscilliscope. This aparent surging could also be as a result of the multimeter internals, in that the ADC could be over shooting, and giving a false reading.
OK, now for the second one. The RC network between the COMP pin on the regulator and ground is used by the internals of the IC for stabilisation, most commonly for frequency and/or phase (which are related) compensation.