Design of an enhanced surface plasmon polaritons (SPPs) based nanostructure for the purpose of beam shaping is discussed. An indentation with depth-tuned grooves is presented to realize the beam shaping and extraordinary transmission. The nanostructure is directly fabricated using focused ion beam (FIB) milling on an Ag thin film coated on quartz with a thickness of 200 nm. A large measurement error is found during geometrical characterization of the nanostructures by use of an atomic force microscope (AFM) working in tapping mode. Apex wearing and 34 deg full cone angle of the probe generate the measurement errors during the characterization of nanostructures with a feature size of 200 nm and below. To solve this problem, an FIB trimmed AFM probe is employed in the geometrical characterization. The results show that the error is improved greatly using the trimmed probe. The desired excitation of the SPPs is derived using an optical fiber coupled CCD spectrometer after the modified geometrical characterization. The designed structure can be used as an optical probe for future inspection and detection use. (c) 2006 Society of Photo-Optical Instrumentation Engineers.