New Paper: Connecting senses: The cross-modal associations between smell and vision in understanding urban environments

In a previous post, I wrote about how geosocial media can be used to map perceived urban smells and how these sensory traces shape people’s perceptions of urban space. Building on that work, we have a new paper entitled “Connecting Senses: The Cross-Modal Associations Between Smell and Vision in Understanding Urban Environments,” written with Ate Poorthuis and Andrew Crooks, and published in Geographical Analysis.

The paper starts from a simple but important mismatch: urban analytics is still often organized around what can be seen, while smell remains harder to observe, classify, and compare across space. Rather than treating street view imagery as a replacement for direct sensory experience, we use it as a proxy for asking a more specific question: when, and under what conditions, can visual cues help us infer perceived smells?

To operationalize this question, we use Mapillary street view imagery from New York City. A subset of images was labeled with predefined smell categories, including Nature, Food, and Transportation & Fuel. These labels were then used to develop a deep learning smell classifier capable of identifying perceived smells from images. We subsequently used image processing models, including ResNet50, VGG16, Inception-V3, MobileNet, and EfficientNet, to extract visual cues and examine how they relate to smell expectations.

In short, the value of this approach lies not in claiming that smell can be fully recovered from images, but in showing where visual evidence can and cannot support multisensory urban analysis. In visually straightforward environments, such as parks or less densely populated areas, visual cues can provide useful signals. In more complex settings, however, overlapping urban odors make the relation between smell and vision much less direct. This tension is precisely what makes the problem interesting.

Abstract:

Smell is a crucial yet understudied sensory dimension in urban environments, bridging tangible elements (e.g., exhaust, flowers) with intangible impacts on emotions, social interactions, and well-being. While geographical and urban research increasingly acknowledges multisensory experiences, much of geospatial analysis still emphasized the visual dimension. This research advances spatial thinking by examining cross-modal associations between smell and vision in urban environments. Specifically, we utilize advanced image processing techniques to extract visual cues from street view imagery (SVI) (i.e., Mapillary) and apply causal analysis to examine their effects on smell expectations recorded from participants. The results show that visual cues can predict smells in straightforward urban settings (e.g., parks or less densely populated areas). However, in complex urban environments, the predictive power of visual cues diminishes as diverse and overlapping scents obscure specific smells, even in visually distinct areas. These findings underscore the importance of a multisensory approach in urban analytics, enhancing our understanding of the interplay between sensory experiences and informing urban design strategies that integrate multiple senses to create engaging and inclusive environments. This is especially important for individuals with sensory impairments, such as anosmia or visual impairments, who rely on other senses to compensate for their perception of urban environments.

Full reference:

Chen, Q., Poorthuis, A., and Crooks, A. T. (2026). Connecting Senses: The Cross-Modal Associations Between Smell and Vision in Understanding Urban Environments. Geographical Analysis, 58(3), e70046. https://doi.org/10.1111/gean.70046

Read more: Project page · Publication page · Code and replication materials