FEATURE POSITIONING ON GOOGLE STREET VIEW PANORAMAS

: Location-based services (LBS) on web-based maps and images have come into real-time since Google launched its Street View imaging services in 2007. This research employs Google Maps API and Web Service, GAE for JAVA, AJAX, Proj4js, CSS and HTML in developing an internet platform for accessing the orientation parameters of Google Street View (GSV) panoramas in order to determine the three dimensional position of interest features that appear on two overlapping panoramas by geometric intersection. A pair of GSV panoramas was examined using known points located on the Library Building of National Chung Hsing University (NCHU) with the root-mean-squared errors of ±0.522m, ±1.230m, and ±5.779m for intersection and ±0.142m, ±1.558m, and ±5.733m for resection in X, Y, and h (elevation), respectively. Potential error sources in GSV positioning were analyzed and illustrated that the errors in Google provided GSV positional parameters dominate the errors in geometric intersection . The developed system is suitable for data collection in establishing LBS applications integrated with Google Maps and Google Earth in traffic sign and infrastructure inventory by adding automatic extraction and matching techniques for points of interest (POI) from GSV panoramas.


INTRODUCTION
The emerging development in microelectronic industry has raised the productivity and performance in mobile mapping devices and the demands in consequent value-added services.Among them the location-based services (LBS) on web-based maps and images have come into real-time since Google launched its Street View imaging services in 2007 (Vincent, 2007).GSV images were taken from a ring of eight cameras plus a fish-eye lens on top for producing the most popular 360-degree panoramic views (Anguelov et al., 2010).As shown in Figure 1, the 4 th generation Street View car was equipped with a R5 system at a height of about 2.5 meters, Global Positioning Systems (GPS) units for vehicle positioning, three laser range scanners for the measuring distances of objects up to 50 meters 180° around the vehicle, and network antennas for scanning 3G/GSM and Wi-Fi hotspots.Each available GSV panorama can be requested in an HTTP URL form using Google Maps Application Programming Interface (Google Maps API), along with the projection type, the geodetic position of the Street View car and its moving direction with respect to the North at the time of image capture.Hence, it provides the user opportunities to determine the spatial position of features that appear on the GSV panoramas.This research employs Google Maps JavaScript API, Google App Engine (GAE) for JAVA, Asynchronous JavaScript And XML (AJAX), and Proj4js (Evenden, 2003) in developing an internet platform for accessing the orientation parameters of GSV panoramas in order to determine the geographic position of any interest features that appear on two overlapping images by intersection, and to determine the position of the exposure center of a panorama from at least three known control points by resection.Extracted 3D positional information of the features from intersection can be packed in Keyhole Markup Language (KML) format and stored in GAE Servlet for future LBS applications integrated with Google Maps and Google Earth.Key elements of the developed platform were introduced in the next section, followed by the results and error analyses from examining known points appeared on the GSV panoramas, and conclusions.

Basics for Google Street View Panorama
Though with composing techniques in veiled, a GSV panorama is a 360° surrounding image generated from mosaicking the eight original images captured by the eight horizontal cameras in the sequences as shown in Figure 2. The percentage of horizontal overlap between adjacent original images is about 28.8% for removing the most distorted outside portion of an original image caused by camera lens.Meanwhile, the panorama was aligned with the heading direction of the vehicle at the center.Each GSV panorama was tile-indexed in a 6 zoom-level image pyramid with the size of 416*208 pixels in level 0, with a scale factor of 2, to 13312*6656 in level 5 for covering the field of view of 360° in horizontal and 180° in vertical.Thus, it is convenient to convert the rectangular image coordinates (row, column) in the developed panorama plane into spherical coordinates (latitude, longitude), and vice versa, in the following equivalent as in the level 5 zooming image:

Parameter
The coordinate accessed from longitude in W getLatLng() m yaw in degree respect to true method for cam the getPov() m elevation (heig be accessed by Web Services.When being p feature or poin and pitch attri angle and vert respectively.T or a specific t may be a label coordinates ac the position of 3.

Error Sources in GSV Positioning
Errors in POI's coordinates from geometric intersection among GSV panoramas may be caused by the following sources: 1. GSV car positioning errors: According to the specification, the worst position errors of the TOPCON IP-S2 GPS/IMU unit for 30 second outage duration (1 epoch/30s) are 0.055m and 0.030m in 2D plane and height direction, respectively and attitude errors of 0.030, 0.030, and 0.075 degrees in roll, pitch, and heading, respectively.The latitude, longitude, and elevation parameters of the GSV panorama may include systematic errors from interpolation.As shown in Table 1, the errors in Google provided GSV positional parameters dominated the errors in intersection, especially in elevation.2. Angular resolution in panoramas: As already shown in Eq.
(1), the conversion from rectangular image coordinates in level 5 zooming panorama into heading and pitch angles is about 1.63 minutes/pixel.The angular resolution will be decreased in the power of 2 when the panorama is zoomed out to lower levels, i.e., 3.25 minutes/pixel for level 4 and 6.5 minutes/pixel for level 3, etc.As a result, when the POI is not carefully located with the highest zoom level of the panorama, the heading and pitch angles of conjugate POI images from the respective panorama center may not be precise enough for deriving reliable results from intersection and resection.3. Distortions of the panoramas: A sphere model was used for merging GSV panoramas from the planar images obtained the horizontal cameras in earlier products.On the other hand, a rectangular plane model is used in the latest Google Maps API version 3.However, the previous GSV products were not regenerated into the rectangle plane model.The panorama may include severe distortions in the upper portions (image from the 9 th camera shooting the sky) and lower portions (rendered from other panoramas) as shown in Figure 7.It would be good to locate POIs on the central portions of ±30 degrees in pitch (vertical angle) according to the vertical field of view of the camera lens.The distortion of image may also happen while the GSV panorama is viewed in different web browsers as shown in Figure 8, in which Google Chrome seems well in reducing the GSV panoramic distortion.4. Small intersection angles between the known points: The intersection angle of a POI from the two adjacent GSV panoramas may be too sharp to cause significant error in intersection (Ghilani & Wolf, 2008).

CONCLUSIONS
This research employs Google Maps JavaScript API and Web Service, GAE for JAVA, AJAX, Proj4js, CSS and HTML in developing an internet platform for accessing the position and orientation parameters of GSV panoramas in order to determine the global positions of POI features that appear on two overlapping images by intersection, and to determine the position of the exposure center of a panorama from at least three known points by resection.Extracted 3-D positional information of the features from intersection were packed in KML format and stored in GAE Servlet for future LBS applications integrated with Google Maps and Google Earth.
Experimental results from positioning known points in a pair of GSV panoramas are ±0.522m,±1.230m, and ±5.779m for intersection and ±0.142m, ±1.558m, and ±5.381m for resection in X, Y, and Z, respectively.Potential error sources in GSV positioning were analyzed and illustrated that the errors in Google provided GSV positional parameters dominate the errors in geometric intersection.
The developed system is suitable for volumetric data collection in establishing LBS applications, in which the positional accuracy is not primarily concerned.Possible applications can be settled in finding and positioning of commercial entities, traffic signs, utility poles and manholes, landmarks, and infrastructures along the streets where GSV panoramas are available.The current state of the GSV positioning system is not complete.Future work on extending its functionalities includes adding automatic feature detection and image matching techniques for object recognition and matching among GSV panoramas.Other features in GSV auxiliary data and Google Maps can also be included to provide the user opportunities to place markers and overlay in the scene, find local business from commercial entity recognition, and lever 3D data and sequence of GSV panoramas for smart navigation.

ACKNOWLEDGEMENTS
The support from the National Science Council, Taiwan, under Grant No. NSC100-2221-E005-075-MY2, was appreciated in improving the developed system in this research.

Figure 7 .
Figure 7. Distortions in the Google Street View Panorama