In this article, I discuss Freud’s theory of fetishism and the most important resources he draws on to elaborate his theory, namely, three texts from the end of the 19th century, Charcot and Magnan’s “Genital Inversions, and Other Sexual Perversions” (1882), Alfred Binet’s “Fetishism in Love” (1887) and von Krafft-Ebing’s Psychopathia Sexualis (1886). I trace their respective theorizing on fetishism, where the attempt to categorize the perversions, a fascination with raw case-material and a focus on degeneracy and heredity as causes are pivotal. I also highlight observations, ideas and concepts in their texts which, while mentioned only in passing, will be crucial for Freud’s and Lacan’s later theories. A close reading of these three texts produces a theory of fetishism that can be summarized as follows: Normal love has fetishistic aspects, but these aspects are harmoniously ordered and result in “normal” sexual intercourse. Therefore, the behaviour of the fetishist cannot be seen as the defining criterion for fetishism. Rather this is a subjective psychical state that has its roots in the association of the awakening of genital excitation with an exterior fact. This association of ideas crystallizes into a fetish, which operates as a sign in a language-based scenario that is instrumental for the regulation of sexual enjoyment in an intense, non-standard way. In fetishists, one finds an ambivalent attitude towards the fetishistic object and a specific relationship to the law. I argue that by the time Freud tackles the question of fetishism, the groundwork has already been done by the early sexologists.