Challenges and Opportunities of Large Language Models in Real-World Machine Learning Applications.

A Workshop Colocated with ECML-PKDD in Turin, Italy and Online in September, 2023

Large Language Models (LLMs) such as GPT-4 have the potential to revolutionize the way we interact with language-based applications in the real world. However, they also come with their set of challenges that need to be addressed to ensure their effective and ethical use. Especially in real-world Machine Learning applications, LLMs have a disruptive potential, but require a carefully designed set of techniques and best practices to avoid a surge in compute cost, risks of producing untruthful or harmful content, and/or an inefficient utilization of their capabilities. This workshop will specifically focus on the opportunities and challenges of LLMs in real-world Machine Learning applications and will cover topics such as synthetic data generation, model evaluation and testing, fairness, debiasing, cold-start, distillation, and prompting. It will also welcome stories of LLM usage in specific applications, such as Chat, Search, Recommendations, Personalization, and/or Spoken Language Understanding.

Important Dates

  • Jun 12th Jun 19th Jun 29th: Paper submission deadline
  • Jul 12th: Author notification
  • Oct 1st: Camera-ready deadline
  • Sep 18th: COLLM 2023 Workshop

Invited Speakers

Alessandro Moschitti, Principal Scientist, Amazon Alexa

Alessandro Moschitti is a Principal Research Scientist of Amazon Alexa AI, where he has been leading the science of Question Answering science since 2018. He obtained his Ph.D. in CS from the University of Rome in 2003, and then did his postdoc at The University of Texas at Dallas for two years. In 2007, he joined the CS Dept. of the University of Trento, Italy. Since 2009 to 2011, he participated to the Jeopardy! Grand Challenge with the IBM Watson Research center, and collaborated with them until 2015. He was a Principal Scientist of the Qatar Computing Research Institute (QCRI) for five years (2013-2018). His expertise concerns theoretical and applied machine learning in the areas of NLP, IR and Data Mining. He is well-known for his work on structural kernels and neural networks for syntactic/semantic inference over text, documented by more than 300 scientific articles. He has received four IBM Faculty Awards, one Google Faculty Award, and five best paper awards. He was the General Chair of EMNLP 2014, a PC co-Chair of CoNLL 2015, and has had a chair role in more than 70 conferences and workshops. He is currently an action/associate editor of ACM Computing Survey and JAIR, and General Chair of EACL 2023. He has led more than 25 research projects, e.g., with MIT CSAIL.

Alex Jaimes, Chief Scientist & SVP of AI, Dataminr

Alex Jaimes is Chief Scientist & SVP of AI at Dataminr and Visiting Professor at Cornell Tech. He is a leader in AI and has built products that are used by millions of people (real-time event detection/emergency response, healthcare, self-driving cars, media, telecomm, etc.) at companies such as Yahoo, Telefónica, IBM, Fuji Xerox, Siemens, AT&T Bell Labs, DigitalOcean, etc. An early voice in Human-Centered AI (Computing), he has over 100 patents and publications in top-tier conferences and journals in AI. He has been featured widely in the press (MIT Tech review, CNBC, Vice, TechCrunch, Yahoo! Finance, etc.). He is a mentor at Endeavor and Techstars, and a member of the advisory board of Digital Divide Data (a non-for-profit that creates sustainable tech opportunities for underserved youth, their families, and their communities in Asia and Africa). He was an expert in the Colombian Government’s Artificial Intelligence Expert Mission which advised the President on AI policies. Alex holds a Ph.D. from Columbia University.

Panel members

Alex Jaimes, Chief Scientist, Chief Scientist

Emanuele Rodolà, Full Professor, Sapienza University of Rome

Kay Rottmann, Senior Applied Scientist, Amazon Alexa

Paul Bennett, Director of Research (LLMs), Spotify

Workshop Organizers

Enrico Palumbo, Spotify, Italy

Davide Bernardi, Amazon Alexa, Italy

Hugues Bouchard, Spotify, Spain

Alessandro Manzotti, Microsoft, Italy

Daniele Amberti, Amazon Alexa, Italy

Program Committee

  • Anjali Shenoy, Research Science Manager, Amazon
  • Anne Schuth, ML Engineering Manager, Spotify
  • Anurag Dwarakanath, Research Science Manager, Amazon
  • Bei Chen, Research Science Manager, Amazon
  • Bianca Scarlini, Research Scientist, Amazon
  • Claudia Hauff, Staff Research Scientist, Spotify
  • Daniele Regoli, Data Scientist, Intesa Sanpaolo
  • Gabriel Benedict, PhD student, University of Amsterdam
  • Geoffrey Liu, Senior Speech Scientist, Microsoft
  • Gustavo Penha, Research Scientist, Spotify
  • Humberto Corona Pampin, Senior Product Manager, Spotify
  • Jacopo Chevallard, Science & Innovation Senior Lead, BLOOM
  • Jose Luis Redondo Garcia, Senior Research Scientist, Spotify
  • Kay Rottmann, Senior Applied Scientist-NLU, Amazon
  • Luca Rubini, Speech Scientist, Nuance communications
  • Luigi Di Caro, Associate Professor, University of Turin
  • Maria Minakova, Research Science Manager, Amazon
  • Melanie Bradford, Research Science Manager, Amazon
  • Piyush Behre, Applied Science Manager, Microsoft
  • Sharman Tan, Applied Scientist, Microsoft
  • Thilina Rajapakse, PhD student, University of Amsterdam
  • Thong Nguyen, PhD student, University of Amsterdam
  • Valerio Basile, Assistant Professor, University of Turin