Perisomatic inhibition provided by a subgroup of GABAergic interneurons plays a critical role in timing the output of pyramidal cells. To test their contribution at the network and the behavioral level, we generated genetically modified mice in which the excitatory drive was selectively reduced either by the knockout of the GluR-D or by conditional ablation of the GluR-A subunit in parvalbumin-positive cells. Comparable cell type-specific reductions of AMPA-mediated currents were obtained. Kainate-induced gamma oscillations exhibited reduced power in hippocampal slices from GluR-D^−/− and GluR-A^PVCre−/− mice. Experimental and modeling data indicated that this alteration could be accounted for by imprecise spike timing of fast-spiking cells (FS) caused by smaller interneuronal EPSPs. GluR-D^−/− and GluR-A^PVCre−/− mice exhibited similar impairments in hippocampus-dependent tasks. These findings directly show the effects of insufficient recruitment of fast-spiking cells at the network and behavioral level and demonstrate the role of this subpopulation for working and episodic-like memory.