... and Tilman Beck, Hendrik Schuff, Anne Lauscher (Universität Hamburg) and Iryna Gurevych (Technische Universität Darmstadt) for the #EACL2024 Social Impact Award!
x.com/UKPLab/statu...
(6/🧵)
1
0
1
0
Hashtag
#EACL2024
Advertisement · 728 × 90
▪️Seven papers authored or co-authored by UKP staff have been accepted for publication at this year's #EACL2024
▪️ Five papers have been accepted for publication at this year's #NAACL2024, the Conference of the North American Chapter of the ACL.
(3/🧵)
1
0
1
0
The papers by Luke Bates, Rachneet Singh Sachdeva, Martin Tutek and Iryna Gurevych are among 7 UKP submissions accepted at #EACL2024. They originate from the project »Fake News and Conspiracy Theories«, part of the ATHENE research area #SeDiTraH:
www.informatik.tu-darmstadt.de/ukp/research...
0
0
0
0
Fighting fake news and misinformation together! 🔎 Our project as part of the German National Research Center for Applied Cybersecurity ATHENE @athenecenter.bsky.social resulted in two accepted papers on the topic at this year's #EACL2024!
Read more: www.athene-center.de/en/news/news...
0
0
1
0
»Sensitivity, Performance, Robustness: Deconstructing the Effect of Sociodemographic Prompting« by Tilman Beck, Hendrik Schuff, Anne Lauscher (Universität Hamburg) and Iryna Gurevych (UKP Lab) was awarded the #EACL2024 Social Impact Award!
More info on the paper in this 🧵: bsky.app/profile/ukpl...
0
0
0
0
Learn more about »M4: Multi-generator, Multi-domain, and Multi-lingual Black-Box Machine-Generated Text Detection« by Yuxia Wang et al., which was awarded the #EACL2024 Resource Paper Award, in this 🧵:
bsky.app/profile/ukpl...
0
0
1
0
At this year's #EACL2024 two papers involving UKP Lab have won awards for Best Resource and Social Impact. Congratulations to everyone involved! 💐
www.tu-darmstadt.de/universitaet...
#EACL2024 #NLProc
0
0
1
0
Presented my poster at #EACL2024 this week! Grateful for the opportunity to share my research and receive insightful feedback. Learned so much and feeling inspired! Huge thanks to the organizers and everyone involved!
1
0
0
0
The award was accepted by Anne Lauscher. More info about the paper can be found in this post: bsky.app/profile/ukpl... #EACL2024
0
0
0
0
We are proud to announce that the contribution »Sensitivity, Performance, Robustness: Deconstructing the Effect of Sociodemographic Prompting« by Tilman Beck, Hendrik Schuff, Anne Lauscher (Universität Hamburg) and Iryna Gurevych (UKP Lab) has just been awarded the #EACL2024 Social Impact Award!
0
0
1
0
And consider getting in touch with the authors Tilman Beck, Hendrik Schuff, Anne Lauscher (Universität Hamburg) and Iryna Gurevych (UKP Lab), if you are interested in more information or an exchange of ideas.
See you this week in Malta!
(9/9) #EACL2024 #NLProc #Prompting #InstructGPT #LLMs
0
0
0
0
Takeaway? Sociodemographic prompting should be used with care ⚠️
If you are interested in the details of this work, you will find more information in the paper & code at:
📄 Paper: arxiv.org/abs/2309.07034
💻 Code: github.com/UKPLab/arxiv...
(8/🧵) #EACL2024 #NLProc
1
0
1
0
Sentiment and toxicity classification benefit from sociodemographic prompting in zero-shot classification.
Could we also use it to identify instances which will likely result in disagreement during annotation? Flan-T5 does a decent job with an avg 0.62 F1.
(7/🧵) #EACL2024 #NLProc
0
0
1
0
Let’s dig deeper into the influence of the model 🕵️: the predictions are dominated by the model family in use!
When prompted with different gender values, InstructGPT places at least 10% of its predictions on label 1 while OPT-IML none, independent of the gender value.
(6/🧵) #EACL2024 #NLProc
0
0
1
0
What are the effects of sociodemographic prompts?
- InstructGPT/OPT-IML are less affected than other models
- Shorter texts lead to more changes
- If the text led to disagreement among annotators, prompting results from different sociodemographic profiles tend to disagree.
(5/🧵) #EACL2024 #NLProc
0
0
1
0
We analyze 17 diverse instruction-tuned LLMs across seven datasets reflecting 4 different subjective NLP tasks (i.e., sentiment, hate speech, toxicity, stance).
(4/🧵) #EACL2024 #NLProc
2
0
1
0
❗ This is very promising, e.g. for dataset augmentation or human survey simulation.
How sensitive are different models to this prompting technique and why? Does it improve classification for subjective tasks, such as hate speech? And how robust is it after all?
(3/🧵) #EACL2024 #NLProc
0
0
1
0
Sociodemographic prompting refers to the idea of enriching a prompt with user profile information such as the gender, race, age, and education level. We expect the model’s output to be aligned with the sociodemographic profile described.
(2/🧵) #EACL2024 #NLProc
0
0
1
0
LLMs are increasingly prompted with different user profiles to solve subjective NLP tasks. What are the factors which determine what the model generates?
Discover it in our #EACL2024 paper – learn more in this 🧵 (1/8).
📰 arxiv.org/abs/2309.07034
#NLProc #Prompting
1
0
1
2
Interested in #ClinicalNLP? Our colleague Tobias Mayer presented his research on Monday at #EACL2024 in 🇲🇹 !
Find out more about "Predicting Client Emotions and Therapist Interventions in Psychotherapy Dialogues" in this thread: bsky.app/profile/ukpl...
0
0
0
0
And consider following the authors Jan Buchmann, Max Eichler, Jan-Micha Bodensohn, Ilia Kuznetsov and Iryna Gurevych (@dfki.bsky.social, Systems@TUDA, @ukplab.bsky.social, @tuda.bsky.social), if you are interested in more information or an exchange of ideas.
See you this week in Malta!
#EACL2024
0
0
0
0
We provide open access to our code and results:
📄 Paper: arxiv.org/abs/2401.17658
💻 Code: github.com/UKPLab/eacl2...
(8/🧵) #EACL2024 #NLProc
0
0
1
0
➡️ Recap: long document transformers implicitly represent document structure. This representation can be enhanced via infusion of structural information, which leads to improvements on downstream tasks.
(7/🧵) #EACL2024 #NLProc
0
0
1
0
Evaluating whether two segment pairs belong to the same document section is a key capability. It is positively correlated with performance on downstream tasks ↗️.
(6/🧵) #EACL2024 #NLProc
0
0
1
0
Does this improved representation of document structure help in downstream tasks?
💯Yes, it does! But the most effective infusion strategy depends on the task.
(5/🧵) #EACL2024 #NLProc
0
0
1
0
What happens if we infuse structural information on node type and depth?
🚀 Accuracy improves on all probing tasks!!
Thanks to an improved representation of document structure.
(4/🧵) #EACL2024 #NLProc
0
0
1
0
We introduce a dataset and novel probing tasks and experiment with LED and LongT5.
Key insight 💡: Long Transformers have implicitly learned to represent document structure during pre-training
(3/🧵) #EACL2024 #NLProc
0
0
1
0
We view documents as graphs
🔵 nodes represent textual elements of different types (e.g. section headings or paragraphs)
➡️ edges represent the hierarchical organization.
(2/🧵) #EACL2024 #NLProc
0
0
1
0
