Projects

Please click on the respective entry to show detailed information. An overview of my DFG-funded project is available at gepris.

Interactive Prompt Optimization with the Human in the Loop for Natural Language Understanding Model Development and Intervention (INPROMPT), 2024-2027)

The paradigm of few-shot or zero-shot learning for the creation of models in algorithmic natural language understanding assumes that little or no annotated text is available for the problem to be solved. Methods in this subject area therefore meet the challenge of relaxing the high data requirements that the optimization of deep neural networks entails. A typical approach is to use pre-trained neural language models and use a prompt to generate a word that describes an instance of text. For example, you can do sentiment polarity classification by entering a text instance such as “The person is very satisfied with the product.” associated with a prompt and check whether the sentence “The product is good” or “The product is bad” results in a higher probability.

Creating such prompts has the advantage that it does not necessarily require technical expertise, but creating good prompts is still not trivial. Existing research has approached the problem from two perspectives: (1) adapting existing language models using (few) annotated data points and manually generated prompt sets, and (2) using data-driven automatic prompt generation.

We combine these two research directions in our project and start with the typical situation in which a language comprehension task is formulated vaguely, a more precise specification is still missing, and no annotated (but certainly non-annotated) texts are available. Our goal is to develop and analyze systems that automatically guide domain experts without technical training in machine learning to create well-functioning prompts.

To do this, we use optimization methods that change prompts iteratively and estimate their quality with the help of a target function. This estimation is based on automatic predictions on text instances, based on the readability of the prompt, and based on the conclusiveness of an explanation of the decision-making. In our project, the objective function based on these factors is not automatically evaluated, but replaced by a “human in the loop”. However, in order to study the problem of iterative optimization of prompts on a larger scale, we also simulate human decisions using automatic approximations of the human objective function.

We expect that our project will significantly improve the transparency of prompt-based models and contribute to the democratization of the use of machine learning algorithms.

  • Role: Applicant
  • Postdoc or Ph.D. student: (open)
  • Funding: DFG (KL 2869/13-1)
The Interplay of Emotions and Convincingness in Argument Mining for NLP (EMCONA, 2024-2027)

Whether an argument is convincing may depend on various factors, for example its logical structure, the clarity of its presentation, but also its emotional connotation. It is known from research in argumentation theory and social psychology that arguments and emotions interact, but this prior knowledge has not yet been (deeply) exploited in natural language processing (NLP) - in contrast, existing work in argument mining for NLP treats emotionality in a shallow manner, typically as an ordinal variable.

In EMCONA, we study the interplay of emotion and convincingness from an NLP perspective and focus on several aspects. We will (1) analyze how emotions are communicated in the context of argumentation. To do so, we will build on top of psychological emotion theories, particularly appraisal theories to estimate the role of societal standards and own goals for the development of emotions. Based on this knowledge and the subsequent development of fine-grained emotion analysis systems, we will (2) analyze the interplay with the convincingness of arguments. This will lead to computational models which jointly represent emotions and argument convincingness, controlled for topic and stance. We will study the interplay between these variables not only in deep learning-based classification settings, but also, (3), in a conditional argument generation setup, which is then (4) evaluated in a user study to understand the boundaries of such systems and where emotions and convincingness are in conflict.

Based on these models and their introspection, we will also, (5), study if they make their decisions following known patterns of communication strategies, for instance “fear-then-relief” (in which fear is induced in an interlocutor for which a solution is subsequently offered) or “door-in-the-face” (which makes a larger initial request than the actual one of interest; in which the emotion of guilt plays an important role) to better understand modeling decision processes.

With this research, we will develop an improved understanding how emotions and convincingness interact in computational argument mining systems. We expect that this knowledge will improve classification approaches for all involved variables. Further, our conditional generation models serve an educational purpose: we ultimately aim for ethical bias-free (fallacy-free) argumentation models that do not exploit unjustified emotions. Our project thus provides the basis to warn (for instance) users in social media in the future when certain argumentation strategies are used, but may also support inexperienced participants in discussions in the creation of high-quality arguments (which make only justified use of emotions). We therefore expect our project to have important societal impact.

  • Role: Applicant
  • Co-Applicant: Steffen Eger (Uni Bielefeld)
  • Postdoc or Ph.D. student: (open)
  • Funding: DFG (KL 2869/12-1)
User’s Choice of Images and Text to Express Emotions in Twitter and Reddit (ITEM, 2024-2027)

Emotions are, next to propositional information, a main ingredient of human interaction. In contrast to information extraction methods, which focus on facts and relations, emotion analysis received comparably little attention and is not yet well understood computationally. Two popular subtasks in emotion analysis in natural language processing are emotion categorization and emotion stimulus detection. For emotion categorization, text is classified into predefined categories, for instance joy, sadness, fear, anger, disgust, and surprise. In stimulus detection, textual segments that describe what happened that caused an associated emotion need to be identified. For instance, the text “I am so happy that my mother will visit me” is associated with joy and the phrase “my mother will visit me” describes the stimulus event.

Next to natural language processing, visual computing has also been applied to emotion categorization, for instance to interpret facial emotion expressions, estimate the impact of artistic peaces on a person, or evaluate depicted events or objects. Further, stimulus detection has seen a similar counterpart to NLP, in which relevant regions in images have been detected. However, no previous work in visual computing exists which puts together whole scenes (with relations between depicted objects and places) for emotion stimulus detection; particularly not informed by emotion theories (which has been done for NLP).

In the project, we advance the state of the art in several directions: (1), we will develop appraisal-theory-based interpretations of images from social media regarding their emotional connotation and stimulus depiction. (2), we will combine this research with our previous work on emotion categorization and stimulus detection in text to develop multimodal approaches. (3), we will do that from both the perspective of the author of a social media post (which emotion is she expressing?) and the intended or probable emotion of a reader (what emotion does an author want to cause, which emotion might a reader feel?).

We will therefore contribute to multimodal emotion analysis and ensure that emotion-related information is not missed or misinterpreted in social media communication because computational models do, so far, not have access to the complete picture. Further, we will answer research questions about how users of social media communicate their emotions, what influences their choices of modality and what the relation between the modalities is.

  • Role: Applicant
  • Co-Investigator: Carina Silberer
  • Postdoc: (open)
  • Ph.D. student: (open)
  • Funding: DFG (KL 2869/11-1)
Neural Language Generation Conditioned on Emotions (NLGCE, 2021-2024)

Intelligent agents as they are implemented in chat bots or assistant systems are successful in solving predefined tasks. However, they lack the capability to behave empathetically, which includes the estimation of the emotional state of the interlocutor and generating appropriate responses. While different research projects exist which focus on the analysis of emotions in human, including the analysis of body postures, textual information, or prosody, there is only a limited amount of work on the generation of emotionally appropriate responses. One reason is that emotions are on the fence between content and style and therefore challenging to be injected in content-wise conditioned responses, based on the dialogue setting or task to be achieved. Further, emotions can be expressed in different ways, including the description of an emotionally connotated event, a bodily reaction, an expression, or a direct report of a subjective feeling. This Ph.D.\ thesis project explores this novel challenge in natural language generation (NLG), namely to condition the generated text not only on the task, content, and dialogue context, but in addition formulate the generated text such that it is appropriate emotionally for the interlocutor. To address the full variety and complexity of emotion expressions, we will integrate psychological theories on event appraisal, emotion regulation and emotional components. We will therefore develop computational methods which will allow intelligent agents to react in the appropriate emotional tone. This will, for the first time, allow smart agents to be perceived as being empathic, not only based on prosody, but also based on content.

  • Role: Supervisor
  • Scholarship Grantee: Yarik Menchaca Resendiz
  • Funding: DAAD
Style Transfer of Psychological Concepts (STPC, 2021-)
Computational Event Evaluation based on Appraisal Theories for Emotion Analysis (CEAT, 2021-2024)

Emotion analysis has typically been formulated as text classification task in which predefined emotion labels are assigned to textual units. The label set commonly follows the set of basic emotions as proposed by Ekman (Anger, Fear, Joy, Surprise, Sadness, Disgust) or Plutchik (adding Trust and Anticipation) or the valence-arousal-dominance model. This constitutes a gap between the state of research in psychology and computational linguistics, as the appraisal theories are widely accepted, but have not been used so far for emotion analysis in text. With CEAT, we fill this gap and develop computational models of the cognitive appraisal of events and, to a lesser degree, of bodily symptoms and action tendencies. To represent the cognitive appraisal, we build on top of Smith/Ellsworth’s (1985) work who show that the variables pleasantness, responsibility, certainty, attention, effort and situational control are sufficient to discriminate between a set of 15 emotions. In this project, we create two approaches to assign these appraisal dimensions to textual event descriptions, firstly by building on top of semantic parsing and secondly in a deep learning setting. Based on these dimensions, we then predict the emotion associated with the textual fragment. This will lead to models that can automatically assign an emotion to an event description, even if no emotion words or self reports of feeling are available.

  • Role: Principle Investigator
  • Other Project Members: Laura Oberlaender, Enrica Troiano, Maximilian Wegge, Fazlourrahman Balouchzahi
  • Funding: DFG (KL 2869/1-2)
  • Selected papers describing the results: 1 2 3 4 5 6 7 8
Automatic Fact-Checking of Biomedical Information in Social Media and Scientific Literature (FIBISS, 2021-2024)

Most research on methods and models for automatic fact checking, which can distinguish misinformation and desinformation from correct information, focus on the news domain. News, including those shared in social media spaces, are checked for their truthfulness.Such methods have not been developed for the biomedical domain yet. Challenges include the richness of (established) sources of information, the complexity of information, as well as the differences between the language of experts and medical laypeople.In this project, we develop information extraction systems for laypeople and expert language, map the extracted information onto each other and finally check their truthfulness, based on established sources.The project combines therefore methods from transfer learning, information extraction, and fact checking for the biomedical domain, especially in social media.

  • Role: Principle Investigator
  • Other Project Members: Amelie Wuehrl, Lara Grimminger, Lynn Greschner, Lyonel Behringer
  • Funding: DFG (KL 2869/5-1)
  • Selected papers describing the results: 1 2 3 4 5

Previous

Emotion and Argument in Digital Information Spaces (EmoArg, 2018-2022)
  • Role: Co-Principle Investigator
  • PIs: Kai Sassenberg (Leibniz-Institut für Wissensmedien) and Sebastian Padó (Uni Stuttgart)
  • Other Project Members: Enrica Troiano (Ph.D. student)
  • Funding: University of Stuttgart
  • Selected papers describing the results: 1 2 3 4

The goal of this project is to develop a better understanding of the impact of emotion on the perception of facts. This aspect is for instance relevant in the distribution of fake news. We further develop methods which are able to separate factual content of statements from the emotional connotation.

More information:

Structured Multi-Domain Emotion Analysis from Text (SEAT, 2018-2021)

Emotion analysis in natural language processings aims at associating text with emotions, for instance with anger, fear, joy, surprise, disgust or sadness. This task extends sentiment analysis, which adds further qualitative value in applications, for instance in social media analysis, in the analysis of fictional stories or news articles.

Existing research has so far mainly focused on the association of text with specific emotion models from psychological research. The development of methods for detecting phrases in text which denote the emotion experiencer (the character or person who feels the emotion), the emotion theme (the cause of the development of an emotion) as well as the modifiers of an emotion (intensifiers and diminishers) has been neglected.

In this project, we aim at filling this gap. We will develop manually annotated corpora from different domains (news, novels, social media) in German and English. Based on these resources, we develop models which are able to automatically recognize and extract such information. We work on different levels: Firstly, we connect words with emotions (with distributional and lexical methods), including grammatical variants. Then, secondly, we analyze these mentions in context with modifiers, the feeler and the theme (cause) of the emotion. Thirdly, we model these information in context, i.e., beyond seperated mentions. All methods will be analyzed regarding their domain and language independence.

  • Role: Principle Investigator
  • Other Project Members: Laura Bostan, Evgeny Kim (Ph.D. students)
  • Funding: DFG (KL 2869/1-1)
  • Selected papers describing the results: 1 2 3 4 5 6 7 8 9 10 11
Comprehensive Modeling of Conversational Contributions in Prose Texts (QUOTE, 2017-2020)

In many kinds of prose texts, both literary or newswire texts, reported speech plays an important role as a source of information aboutcharacters, their attitudes, and their relationships. Going further,such information can aid in the analysis of patterns of behavior and theconstruction of social networks.While readers do not have any problem in assembling representations forcomplete situations from individual instances of reported speech, thisis still a challenging task for computers. Current state of the artmethods are generally organized as “pipelines” which start fromindividual instances of reported speech and proceed incrementally tomore global properties of the situation or characters. Since individualinstances of reported speech are often short and uninformative, apipeline procedure often causes prediction errors which cannot berectified in retrospect.In this project, we develop joint inference methods to model the variousaspects of reported speech (who is the speaker? the hearer? What is thecontent? What is the relationship between speaker and hearer?) togetherinstead of individually. The resulting joint model takes account of theinterdependencies between these aspects. Thus, information from thedifferent aspects can complement each other. The result of this part ofthe project is a solid starting place (in terms of natural languageprocessing methods) for the application of such methods for theautomatic analysis of reported speech in digital humanities and socialsciences.This algorithmic goal is complemented by a goal from corpus andcomputational linguistics, namely elucidating the relationship betweenreported speech and other aspects of semantic analysis. In particular,there appears to be a close relationship between reported speech and (asubset) of semantic roles. Yet, no comprehensive formal analysis hasbeen carried out so far. We will provide a linguistic characterizationof the relationship and exploit it algorithmically to further improvethe recognition of reported speech as discussed above. The results ofthis part of the project is the (at least partial) consolidation of twostrands of research that have largely been treated as independent sofar.

  • Role: Co-Principle Investigator
  • Principle Investigator: Sebastian Padó
  • Other Project Members: Sean Papay (Ph.D. student)
  • Funding: DFG (PA 1956/4-1)
  • Selected papers describing the results: 1 2 3 4 5
PSINK - Automatische Erstellung einer Wissensbasis zur Unterstuetzung der Translation von der praeklinischen Forschung in die klinische Anwendung bei Rueckenmarksverletzungen (PSINK, 2016-2020)
  • Role: Co-Principle Investigator and Proposal Author
  • Principle Investigator: Philipp Cimiano and Hans-Werner Mueller
  • Other Project Members: Nicole Brazda (Co-PI), Hendrik ter Horst, Veronica Estrada, Jessica Schira, Christian Ohmann
  • Former Project Members: Matthias Hartung
  • Funding: BMBF, i:DSem

We develop automatic information extraction methods to populate a database of preclinical experiments in the domain of spinal cord injuries.

Partners are:

More information:

Center for Reflective Text Analysis (CRETA, 2016--2021)

The Chair of Theoretical Computational Linguistics is a partner in this BMBF-funded project in which we work on emotion analysis for literature analysis. Partners at University of Stuttgart are: Institut für Literaturwissenschaft / Germanistische Mediävistik, Institut für Visualisierung und Interaktive Systeme, Historisches Institut / Landesgeschichte, Institut für Sozialwissenschaften / Internationale Beziehungen und Europäische Integration, Institut für Maschinelle Sprachverarbeitung, Institut für Philosophie / Wissenschaftstheorie und Technikphilosophie, Institut für Literaturwissenschaft / Neuere Deutsche Literatur, Institut für Linguistik / Romanistik, Institut für Literaturwissenschaft / Digital Humanities, Stuttgart Research Centre for Text Studies

More information, including publications:

Confidence Estimation for Biomedical Information Extraction (KABI, 2016--2018)
  • Role: Principle Investigator
  • Other project members: Camilo Thorne (Postdoc)
  • Funding: MWK Baden-Württemberg and University of Stuttgart

In the Life Sciences, most information is only available in free text in scientific publications. Automatic methods to extract such knowledge and to provide it in structured databases is challenged by a dilemma: Especially if potentially new information is detected in text, it is unclear if this information is actually correct or if it is wrongly extracted, for instance because the text is formulated in an uncommon way. In this project, methods will be developed which help to estimate the reliability of extracted information from biomedical publications.

Recipe Classification (2016)
  • Role: Principle Investigator
  • Other project members: Christian Scheible, Hanna Kicherer
  • Funding: Chefkoch GmbH
It's OWL (2013 – 2014)
  • Role: Postdoc in Project MMI
  • Funding: German Federal Ministry of Research and Education (BMBF)
Sentiment Analysis for Distance Education Evaluation (SADE, 2014)
  • Role: Principle Investigator
  • Funding: Online Akademie GmbH
+Spaces (2011 – 2012)
Aneurist (2006 – 2010)
Learning and Inference Platform (2006 – 2008)
  • Role: Ph.D. student
  • Funding: Fraunhofer and Max-Planck Societies

  • Other partners: MPG-FhG Collaboration Project with Fraunhofer FIRST, SCAI, IAIS, ITWM and Max-Planck Friedrich Miescher Laboratory, Institute for biological Cybernetics, Institute for Informatics, Institute for Molecular Genetics

  • Learning and Inference Platform