[DL] [AIxIA] Spotlight Seminar on AI - Bernardo Magnini - February 27, 2025

[Luciano Serafini]

[* Apologies in case of multiple posting *]

The Italian Association for Artificial Intelligence, in collaboration with the AISB (Society for the Study of Artificial Intelligence and Simulation of Behaviour, UK) is pleased to announce the next seminar of its Spotlight Seminars on AI initiative:

February 27  – 5:00 P.M. (CET)
Title: Rethinking NLP Evaluation in the Age of LLMs: Lessons from Benchmarking Italian
Speaker: Bernardo Magnini, FBK, Italy

Link Streaming: https://www.youtube.com/@AixiaIt/streams <https://www.google.com/url?q=https://www.youtube.com/@AixiaIt/streams&source=gmail-imap&ust=1740484109000000&usg=AOvVaw2IZkp361yEQOoCnPZqonJi> 

Abstract:  Large Language Models (LLMs) are now at the core of most NLP applications, mainly because of their strong performance and their adaptability to different tasks and languages. However, despite their widespread use, evaluating LLMs is still an active area of research, and a debate about methodologies is ongoing. Several issues are under discussion, including competence-oriented and task-oriented approaches; how to balance prompt naturalness and effectiveness; investigate the role of multiple prompts in evaluation; considering both multiple-choice and generative tasks along with the most appropriate metrics for each; and comparing zero-shot and few-shot settings taking into consideration execution performance. To be more concrete, I will report examples and lessons learned from developing an LLM benchmark for the Italian language.

Bio: Bernardo Magnini is senior researcher at FBK (Trento, Italy), and responsible of the NLP research group. His interests are in the field of Computational Linguistics, particularly lexical semantics and lexical resources, question answering, textual entailment, and conversational agents, areas in which he has published more than 300 scientific papers. He has co-chaired several events, including EVALITA, the evaluation campaign for both NLP and speech tools for the Italian language, CLIC-it 2014 (the first Italian conference on Computational Linguistics), AI*IA 2018 (the 17th International Conference of the Italian Association for Artificial Intelligence) and ACL 2022, the 60th Annual Meeting of the Association for Computational Linguistics. He has been contract professor at the University of Trento, Bolzano and Pavia, and President of the Italian Association for Computational Linguistics (AILC) from 2015 to 2022.
	
The joint AI*IA/AISB Committee of the Spotlight Seminars on AI, 

Giuseppe De Giacomo
Floriana Grasso
Antonio Lieto
Bertie Muller
Luciano Serafini


-- 
-- 
Hai ricevuto questo messaggio in quanto sei iscritto al gruppo Google
"AIxIA mailing list".
Per mandare un messaggio a questo gruppo, invia una email a
aixia at aixia.it
Solo gli iscritti possono inviare messaggi.
Per annullare l'iscrizione a questo gruppo, invia un'email a aixia+unsubscribe at aixia.it
Per iscriversi visita
http://groups.google.com/a/aixia.it/group/aixia <https://www.google.com/url?q=http://groups.google.com/a/aixia.it/group/aixia&source=gmail-imap&ust=1740484109000000&usg=AOvVaw2jTTAg5x4PC-74YaYTu03_>
Puoi iscriverti con il tuo account aixia.it o gmail.com
 
You received this message because you have subscribed the Google group "AIxIA mailing list".
To send a message to this group, send an email to aixia at aixia.it
Only the subscribers can send to this group.
To cancel the subscription to this group, send an email to aixia+unsubscribe at aixia.it
To subscribe, visit 
http://groups.google.com/a/aixia.it/group/aixia <https://www.google.com/url?q=http://groups.google.com/a/aixia.it/group/aixia&source=gmail-imap&ust=1740484109000000&usg=AOvVaw2jTTAg5x4PC-74YaYTu03_>
You can subscribe with your aixia.it or gmail,com account
To unsubscribe from this group and stop receiving emails from it, send an email to aixia+unsubscribe at aixia.it <mailto:aixia+unsubscribe at aixia.it>.

-- 
--
Le informazioni contenute nella presente comunicazione sono di natura 
privata e come tali sono da considerarsi riservate ed indirizzate 
esclusivamente ai destinatari indicati e per le finalità strettamente 
legate al relativo contenuto. Se avete ricevuto questo messaggio per 
errore, vi preghiamo di eliminarlo e di inviare una comunicazione 
all’indirizzo e-mail del mittente.

--
The information transmitted is 
intended only for the person or entity to which it is addressed and may 
contain confidential and/or privileged material. If you received this in 
error, please contact the sender and delete the material.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.zih.tu-dresden.de/pipermail/dl/attachments/20250218/8712f580/attachment-0001.htm>