You don’t need to recompile the entire image, but as far as I gather, starting to recompile the entire image is the easiest way of obtaining a cross-compiler. You can stop after that.