So you have to set the speed of the camera pretty slow in order to use your arrow keys for the panning. Easiest way to set the speed of the camera movement is by zooming into the object (preferably use a planet as reference to set the panning speed), close enough to get the relative speed you want. From there, you can use your arrow keys to move at a pace you want to pan.
Note: If you zoom in or out, you will lose the speed of the camera movement after you obtain the desired speed. Just trial and error that I've done in the past.
I tried not to use my mouse so much. It's never precise nor smooth.
I hope this helps. 
