Which software was this, because widening orbits make no real sense, at least not at that scale. When you have an object with a given orbital vector around a mass and no further energy put in there's really only two possible solutions, a parabolic orbit or a hyperbolic orbit (an impact is still a parabolic orbit. It is simply one in which periapsis is beneath the surface of the major body).
As for how the drive 'knows', it 'knows' the same way a jump drive 'knows' it is too close to a body. It's just an aspect of the physics. I assume the drive works off of any sufficiently massive body that is within range. The problem is that the range is really very short (I think the limit is 10D) and so you almost never get two objects of sufficient mass that are close enough to one another for a drive to be in range of both.
I do realize that Diameters is not really a good way to deal with gravity but it's how Traveller does it for the sake of convenience. If you really want to get nerdy we could assume that 10D is relative to Earth and so what really happens is the drive loses effectiveness when the force of gravity is less than .01 G's.
I didn't catch the question the first time, the software is Mathematica. I set up a system of ordinary differential equations as the equations of motion and either use Mathematica's RK integrator or a self written velocity verlet integrator.
This appeared in the news today, which talks about a near-earth companion asteroid that is not gravitationally bound to the earth - is in an independent solar orbit around the sun.
As a further comment to some of the other comments in this thread that didn't seem to understand that some of my plots were earth-centric, I'll use this an an example too:
I grabbed the October 14 2017 state vector for the asteroid referenced in that article, inserted it into the simulation, and got this:
Replotting the same simulation centered on the Earth gives this (the plot runs for 2 years):
(red = the moon, there is a green dot in the center of the second plot representing the earth, black is venus which makes a brief appearance, and blue is the companion asteroid. Note the interesting "orbit" that the companion appears to make as if it were somehow bound to earth, even though it is not. That was the same situation as the spiral that the traveller spacecraft appeared to make.
To answer the other aspect of this, why should the orbit of the spacecraft change at all, of course an object that is on the surface of the earth (at the equator) which has the angular momentum of any object at the surface of the earth will have a different amount of energy than if it were only in solar orbit.
Think of a spacecraft doing a close slingshot flyby of jupiter - if you suddenly turn off jupiter's gravity at the spacecraft's closest approach it will end up in a different solar orbit than if you left the gravity on the whole time. Exact same thing.