<html><head><meta http-equiv="content-type" content="text/html; charset=utf-8"></head><body style="overflow-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;"><br><div style="line-height: 1.2; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size: 11pt; font-family: Arial, sans-serif; color: rgb(0, 0, 0); background-color: transparent; font-variant-numeric: normal; font-variant-east-asian: normal; font-variant-alternates: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">[* Apologies in case of multiple posting *]</span></div><br><div style="line-height: 1.2; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size: 11pt; font-family: Arial, sans-serif; color: rgb(0, 0, 0); background-color: transparent; font-variant-numeric: normal; font-variant-east-asian: normal; font-variant-alternates: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">The Italian Association for Artificial Intelligence, in collaboration with the AISB (Society for the Study of Artificial Intelligence and Simulation of Behaviour, UK) is pleased to announce the next seminar of its Spotlight Seminars on AI initiative:</span></div><br><div style="line-height: 1.2; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size: 11pt; font-family: Arial, sans-serif; color: rgb(0, 0, 0); background-color: transparent; font-variant-numeric: normal; font-variant-east-asian: normal; font-variant-alternates: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">February 27 – 5:00 P.M. (CET)</span></div><div style="line-height: 1.2; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size: 11pt; font-family: Arial, sans-serif; color: rgb(0, 0, 0); background-color: transparent; font-variant-numeric: normal; font-variant-east-asian: normal; font-variant-alternates: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">Title: </span><span style="font-size: 11pt; font-family: Arial, sans-serif; color: rgb(34, 34, 34); font-variant-numeric: normal; font-variant-east-asian: normal; font-variant-alternates: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">Rethinking NLP Evaluation in the Age of LLMs: Lessons from Benchmarking Italian</span></div><div style="line-height: 1.2; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size: 11pt; font-family: Arial, sans-serif; color: rgb(0, 0, 0); background-color: transparent; font-variant-numeric: normal; font-variant-east-asian: normal; font-variant-alternates: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">Speaker: Bernardo Magnini, FBK, Italy</span></div><br><div style="line-height: 1.2; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size: 11pt; font-family: Arial, sans-serif; color: rgb(0, 0, 0); background-color: transparent; font-variant-numeric: normal; font-variant-east-asian: normal; font-variant-alternates: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">Link Streaming: </span><a href="https://www.google.com/url?q=https://www.youtube.com/@AixiaIt/streams&source=gmail-imap&ust=1740484109000000&usg=AOvVaw2IZkp361yEQOoCnPZqonJi"><span style="font-size: 11pt; font-family: Arial, sans-serif; color: rgb(17, 85, 204); background-color: transparent; font-variant-numeric: normal; font-variant-east-asian: normal; font-variant-alternates: normal; font-variant-position: normal; text-decoration-line: underline; text-decoration-skip-ink: none; vertical-align: baseline; white-space-collapse: preserve;">https://www.youtube.com/@AixiaIt/streams</span></a><span style="font-size: 11pt; font-family: Arial, sans-serif; color: rgb(0, 0, 0); background-color: transparent; font-variant-numeric: normal; font-variant-east-asian: normal; font-variant-alternates: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"> </span></div><br><div style="line-height: 1.2; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size: 11pt; font-family: Arial, sans-serif; color: rgb(34, 34, 34); background-color: transparent; font-variant-numeric: normal; font-variant-east-asian: normal; font-variant-alternates: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">Abstract: </span><span style="font-size: 11pt; font-family: Arial, sans-serif; color: rgb(34, 34, 34); font-variant-numeric: normal; font-variant-east-asian: normal; font-variant-alternates: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">Large Language Models (LLMs) are now at the core of most NLP applications, mainly because of their strong performance and their adaptability to different tasks and languages. However, despite their widespread use, evaluating LLMs is still an active area of research, and a debate about methodologies is ongoing. Several issues are under discussion, including competence-oriented and task-oriented approaches; how to balance prompt naturalness and effectiveness; investigate the role of multiple prompts in evaluation; considering both multiple-choice and generative tasks along with the most appropriate metrics for each; and comparing zero-shot and few-shot settings taking into consideration execution performance. To be more concrete, I will report examples and lessons learned from developing an LLM benchmark for the Italian language.</span></div><div style="line-height: 1.2; margin-top: 0pt; margin-bottom: 0pt;"><br></div><div style="line-height: 1.2; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size: 11pt; font-family: Arial, sans-serif; color: rgb(0, 0, 0); background-color: transparent; font-variant-numeric: normal; font-variant-east-asian: normal; font-variant-alternates: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">Bio: </span><span style="font-size: 11pt; font-family: Arial, sans-serif; color: rgb(34, 34, 34); font-variant-numeric: normal; font-variant-east-asian: normal; font-variant-alternates: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">Bernardo Magnini is senior researcher at FBK (Trento, Italy), and responsible of the NLP research group. His interests are in the field of Computational Linguistics, particularly lexical semantics and lexical resources, question answering, textual entailment, and conversational agents, areas in which he has published more than 300 scientific papers. He has co-chaired several events, including EVALITA, the evaluation campaign for both NLP and speech tools for the Italian language, CLIC-it 2014 (the first Italian conference on Computational Linguistics), AI*IA 2018 (the 17th International Conference of the Italian Association for Artificial Intelligence) and ACL 2022, the 60th Annual Meeting of the Association for Computational Linguistics. He has been contract professor at the University of Trento, Bolzano and Pavia, and President of the Italian Association for Computational Linguistics (AILC) from 2015 to 2022.</span></div><p dir="ltr" style="line-height: 1.2; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size: 11pt; font-family: Arial, sans-serif; color: rgb(0, 0, 0); background-color: transparent; font-variant-numeric: normal; font-variant-east-asian: normal; font-variant-alternates: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;"><span style="text-wrap: nowrap;"> </span></span></p><div style="line-height: 1.2; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size: 11pt; font-family: Arial, sans-serif; color: rgb(0, 0, 0); background-color: transparent; font-variant-numeric: normal; font-variant-east-asian: normal; font-variant-alternates: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">The joint AI*IA/AISB Committee of the Spotlight Seminars on AI, </span></div><br><div style="line-height: 1.2; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size: 11pt; font-family: Arial, sans-serif; color: rgb(0, 0, 0); background-color: transparent; font-variant-numeric: normal; font-variant-east-asian: normal; font-variant-alternates: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">Giuseppe De Giacomo</span></div><div style="line-height: 1.2; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size: 11pt; font-family: Arial, sans-serif; color: rgb(0, 0, 0); background-color: transparent; font-variant-numeric: normal; font-variant-east-asian: normal; font-variant-alternates: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">Floriana Grasso</span></div><div style="line-height: 1.38; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size: 11pt; font-family: Arial, sans-serif; color: rgb(0, 0, 0); background-color: transparent; font-variant-numeric: normal; font-variant-east-asian: normal; font-variant-alternates: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">Antonio Lieto</span></div><div style="line-height: 1.38; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size: 11pt; font-family: Arial, sans-serif; color: rgb(0, 0, 0); background-color: transparent; font-variant-numeric: normal; font-variant-east-asian: normal; font-variant-alternates: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">Bertie Muller</span></div><div style="line-height: 1.2; margin-top: 0pt; margin-bottom: 0pt;"><span style="font-size: 11pt; font-family: Arial, sans-serif; color: rgb(0, 0, 0); background-color: transparent; font-variant-numeric: normal; font-variant-east-asian: normal; font-variant-alternates: normal; font-variant-position: normal; vertical-align: baseline; white-space-collapse: preserve;">Luciano Serafini</span></div><br><div><br class="webkit-block-placeholder"></div>
-- <br>
-- <br>
Hai ricevuto questo messaggio in quanto sei iscritto al gruppo Google<br>
"AIxIA mailing list".<br>
Per mandare un messaggio a questo gruppo, invia una email a<br>
aixia@aixia.it<br>
Solo gli iscritti possono inviare messaggi.<br>
Per annullare l'iscrizione a questo gruppo, invia un'email a aixia+unsubscribe@aixia.it<br>
Per iscriversi visita<br>
<a href="https://www.google.com/url?q=http://groups.google.com/a/aixia.it/group/aixia&source=gmail-imap&ust=1740484109000000&usg=AOvVaw2jTTAg5x4PC-74YaYTu03_">http://groups.google.com/a/aixia.it/group/aixia</a><br>
Puoi iscriverti con il tuo account aixia.it o gmail.com<br>
<br>
You received this message because you have subscribed the Google group "AIxIA mailing list".<br>
To send a message to this group, send an email to aixia@aixia.it<br>
Only the subscribers can send to this group.<br>
To cancel the subscription to this group, send an email to aixia+unsubscribe@aixia.it<br>
To subscribe, visit <br>
<a href="https://www.google.com/url?q=http://groups.google.com/a/aixia.it/group/aixia&source=gmail-imap&ust=1740484109000000&usg=AOvVaw2jTTAg5x4PC-74YaYTu03_">http://groups.google.com/a/aixia.it/group/aixia</a><br>
You can subscribe with your aixia.it or gmail,com account<br>
To unsubscribe from this group and stop receiving emails from it, send an email to <a href="mailto:aixia+unsubscribe@aixia.it">aixia+unsubscribe@aixia.it</a>.<br>
</body></html>
<br>
<div style="font-family:Arial,Helvetica,sans-serif"><span style="font-size:x-small">--</span></div><div><div><font size="1"><font face="Arial, Helvetica, sans-serif">Le informazioni contenute nella presente comunicazione sono di natura </font><span style="font-family:Arial,Helvetica,sans-serif">privata e come tali sono da considerarsi riservate ed indirizzate </span><span style="font-family:Arial,Helvetica,sans-serif">esclusivamente ai destinatari indicati e per le finalità strettamente </span><span style="font-family:Arial,Helvetica,sans-serif">legate al relativo contenuto. Se avete ricevuto questo messaggio per </span><span style="font-family:Arial,Helvetica,sans-serif">errore, vi preghiamo di eliminarlo e di inviare una comunicazione </span><span style="font-family:Arial,Helvetica,sans-serif">all’indirizzo e-mail del mittente.</span></font></div></div><div style="font-family:Arial,Helvetica,sans-serif"><font face="Arial, Helvetica, sans-serif" size="1">--</font></div><div style="font-family:Arial,Helvetica,sans-serif"><font size="1">The information transmitted is intended only for the person or entity to which it is addressed and may contain confidential and/or privileged material. If you received this in error, please contact the sender and delete the material.</font></div>