In human language several ambiguities cannot be resolved without simultaneously reasoning about an associated context. Often, the context can be best catpured from the visual scene referred by the sentence. If we consider the sentence “I take a photograph of a chimpanzee in my pajamas”, looking at language alone, it is unclear if it is the person or the chimpanzee wearing the pajamas. In this dissertation we focus on the contextual effects on semantics: on the one hand we investigate such contextual effects on a disambiguation task using neural computational simulation; on the other hand we propose a novel context sensitive cognitive account of similarity. Going a little more in detail in our disambiguation task, provided with a sentence, admitting two or more candidate interpretations, and an image that depicts the content of the sentence, it is required to choose the correct interpretation of the sentence depending on the image’s content. Thus we address the problem of selecting the interpretation of an ambiguous sentence matching the content of a given image. This type of inference is frequently called for in human communication that occurs in a visual environment, and is crucial for language acquisition, when much of the linguistic content refers to the visual surroundings of the child [8, 11]. This kind of task is also fundamental to the problem of grounding vision in language, by focusing on phenomena of linguistic ambiguity, which are prevalent in language, but typically overlooked when using language as a medium for expressing understanding of visual content. Due to such ambiguities, a superficially appropriate description of a visual scene may in fact not be sufficient for demonstrating a correct understanding of the relevant visual content. Regarding our new contextual account of similarity, we will suggest that most of the traditional similarity models which have been proposed over the years can converge on a generalized model of similarity in which the context plays a fundamental role in order to overcome all the criticisms raised over the years to each of the traditional similarity models. From the neurocomputational point of view, our models are based on the Eliasmith’s Neural Engineering Network (NEF) [27] and Nengo1, the python library which serves as an implementation of the NEF. The basic semantic component within NEF is the so- called Semantic Pointer Architecture (SPA) [129], which determines how the concepts are represented as dymanic neural assemblies.

Neural Models of Contextual Semantic Disambiguation

PAVONE, Arianna Maria
2021-03-10

Abstract

In human language several ambiguities cannot be resolved without simultaneously reasoning about an associated context. Often, the context can be best catpured from the visual scene referred by the sentence. If we consider the sentence “I take a photograph of a chimpanzee in my pajamas”, looking at language alone, it is unclear if it is the person or the chimpanzee wearing the pajamas. In this dissertation we focus on the contextual effects on semantics: on the one hand we investigate such contextual effects on a disambiguation task using neural computational simulation; on the other hand we propose a novel context sensitive cognitive account of similarity. Going a little more in detail in our disambiguation task, provided with a sentence, admitting two or more candidate interpretations, and an image that depicts the content of the sentence, it is required to choose the correct interpretation of the sentence depending on the image’s content. Thus we address the problem of selecting the interpretation of an ambiguous sentence matching the content of a given image. This type of inference is frequently called for in human communication that occurs in a visual environment, and is crucial for language acquisition, when much of the linguistic content refers to the visual surroundings of the child [8, 11]. This kind of task is also fundamental to the problem of grounding vision in language, by focusing on phenomena of linguistic ambiguity, which are prevalent in language, but typically overlooked when using language as a medium for expressing understanding of visual content. Due to such ambiguities, a superficially appropriate description of a visual scene may in fact not be sufficient for demonstrating a correct understanding of the relevant visual content. Regarding our new contextual account of similarity, we will suggest that most of the traditional similarity models which have been proposed over the years can converge on a generalized model of similarity in which the context plays a fundamental role in order to overcome all the criticisms raised over the years to each of the traditional similarity models. From the neurocomputational point of view, our models are based on the Eliasmith’s Neural Engineering Network (NEF) [27] and Nengo1, the python library which serves as an implementation of the NEF. The basic semantic component within NEF is the so- called Semantic Pointer Architecture (SPA) [129], which determines how the concepts are represented as dymanic neural assemblies.
10-mar-2021
File in questo prodotto:
File Dimensione Formato  
Tesi-Arianna-Pavone.pdf

accesso aperto

Tipologia: Tesi di dottorato
Licenza: Creative commons
Dimensione 12.75 MB
Formato Adobe PDF
12.75 MB Adobe PDF Visualizza/Apri
Pubblicazioni consigliate

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11570/3192169
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus ND
  • ???jsp.display-item.citation.isi??? ND
social impact