Ytdyklly

I'm wondering if it is possible for a planet to be impossible to orbit stably over a significant length of time. The shorter the length of time the planet can be orbited for, the better. It should be possible to land on the planet, just not to maintain an orbit around it without the requirement for near-constant fuel expenditure for stationkeeping purposes.

What mechanism could cause the planet to be almost impossible to orbit for more than a few orbital periods?

The simplest answer that comes to my mind is that the planet is surrounded by a cloud of debris (micrometeorites).

These are usually quite small, so can be detected only some minutes before the impact and follow very different orbits, with different inclinations compared to one anothe. So a ship in orbit should continuously maneuver to keep itself safe from impacts, while a landing ship could easily find a free window to cross safely the cloud and land.

The cloud of debris should be recent: in the long run, such a cloud would probably gradually fall onto the surface of the planet because of kinetic energy loss for high atmosphere friction or impacts among the micrometeorites.

Consider also that it is not necessary to have the orbits of the planet "covered" by these micrometeorites: I think that just some tens of thousands of these object, with enough random trajectories, could make orbiting around the planet a risky choice.

Of course the danger would be high only near the planet: probably it would be still possible to orbit it, even from very far.

EDIT

As Fay pointed, a similar scenario is called Kessler syndrome. In such case, the lower orbits would be safer for a ship to orbit (but it would still need to periodically burn its engines in order to correct its orbit because of air drag)

Have it orbit very close to a massive body

What you want is a very small Hill sphere, which is the region of space where a particular body's gravitational influence dominates. On the other hand, the celestial body probably will have to become a moon.

If the mass of the smaller body (e.g. Earth) is ${displaystyle}$ m, and it orbits a heavier body (e.g. Sun) of mass ${displaystyle M}$ with a semi-major axis ${displaystyle a}$ and an eccentricity of ${displaystyle e}$, then the radius ${displaystyle r_{mathrm {H} }} $ of the Hill sphere of the smaller body (e.g. Earth) calculated at pericenter is, approximately

${displaystyle r_{mathrm {H} }approx a(1-e){sqrt[{3}]{frac {m}{3M}}}.} $

The above equation means if a satellite orbits a more massive body, it will have a smaller Hill sphere even if the satellite orbit stays the same. There is a limit on how close bodies can orbit each other however. Its called the Roche limit, and objects closer than this limit gets ripped apart by tidal forces.

The Roche limit for a rigid spherical satellite is the distance, ${displaystyle d}$, from the primary at which the gravitational force on a test mass at the surface of the object is exactly equal to the tidal force pulling the mass away from the object:

...

${displaystyle d=R_{m}left(2{frac {M}{m}}right)^{frac {1}{3}}}$

where ${displaystyle R_{m}}$ is the radius of the secondary, ${displaystyle M}$ is the mass of the primary, and ${displaystyle m}$ is the mass of the secondary.

(edited for formatting clarity)

So put your planet-like moon close to the parent body and make the parent body big. That might cause a host of other problems though depending on your story such as massive amounts of radiation, high delta-v cost to get to and from the planet, etc, depending on the details.

To orbit a planet the main factor for orbiting a Planet is Mass (gravity of an object) and Speed. The best way would be a multi planet solar system, where the planets orbits are getting close enough to interfere with each others gravity, so orbiting would only be possible for a short time.
However I don't know how the planets themselves could stay in their orbits and not change orbits themselves.