In the realm of autonomous systems and smart-city initiatives, accurately detecting and localizing pole-like objects (PLOs) such as electrical poles and traffic signs has become crucial. Despite their significance, the diverse nature of PLOs complicates their accurate recognition. Point cloud data and 3D deep learning models offer a promising approach to PLO localization under varied lighting, addressing issues faced by camera systems. However, the distinct characteristics of different street scenes worldwide require infeasibly extensive training data for satisfactory results because of the nature of deep learning. This prohibitively increases the cost of lidar data capture and annotation. This paper introduces a novel few-shot learning framework for the classification of outdoor point cloud objects, leveraging a minimalistic approach that requires only a single support sample for effective classification. Central to our methodology is the development of Pole-NN, a Non-parametric Network that efficiently distinguishes between various PLOs and other road assets without the need for extensive training datasets traditionally associated with deep learning models. Additionally, we present the Parkville-3D Dataset, an annotated point cloud dataset we have captured and labelled, which addresses the notable scarcity of fine-grained PLO datasets. Our experimental results demonstrate the potential of our approach to utilize the intrinsic spatial relationships within point cloud data, promoting a more efficient and resource-conscious strategy for PLO classification.