In this paper we propose a novel workflow for the estimation of global image orientations given relative orientations between pairs of overlapping images. Our approach is convex and independent on initial values. First, global rotations are estimated in a relaxed semidefinite program (SDP) and refined in an iterative least squares adjustment in the tangent space of SO(3). A critical aspect is the handling of outliers in the relative orientations. We present a novel heuristic graph based approach for filtering the relative rotations that outperforms state-of-the-art robust rotation averaging algorithms. In a second part we make use of point-observations, tracked over a set of overlapping images and formulate a linear homogeneous system of equations to transfer the scale information between triplets of images, using estimated global rotations and relative translation directions. The final step consists of refining the orientation parameters in a robust bundle adjustment. The proposed approach handles outliers in the homologous points and relative orientations in every step of the processing chain. We demonstrate the robustness of the procedure on synthetic data. Moreover, the performance of our approach is illustrated on real world benchmark data.