1. Execute Convert_Prepare every step.
If this is your camera code...
d3d_set_projection_ext(x,y,16,xto,yto,zto,0,0,1,45,1.6,1,32000);... then these are the arguments required for Convert_Prepare:
Convert_Prepare(x,y,16,xto,yto,zto,0,0,1,45,1.6); // <- same arguments as d3d_set_projection_ext except for znear and zfar2. Then use Convert_3d or Convert_2d depending on what you'd want to convert.
If all else fails, compare your code to the code in my example and see what you did different (or forgot).