Accepted to TMLR 2023.
How well do large language models perform on an unnatural in-context learning task?
< papers
unnatural in-context learning task
Jun 9, 2022
@Aarohi Srivastava, @Abhinav Rastogi, @Abhishek Rao, @Abu Awal Md Shoeb, @Abubakar Abid, @Adam Fisch, @Adam R. Brown, @Adam Santoro, @Aditya Gupta, @Adrià Garriga-Alonso, @Agnieszka Kluska, @Aitor Lewkowycz, @Akshat Agarwal, @Alethea Power, @Alex Ray, @Alex Warstadt, @Alexander W. Kocurek, @Ali Safaya, @Ali Tazarv, @Alice Xiang, @Alicia Parrish, @Allen Nie, @Aman Hussain, @Amanda Askell, @Amanda Dsouza, @Ambrose Slone, @Ameet Rahane, @Anantharaman S. Iyer, @Anders Andreassen, @Andrea Madotto, @Andrea Santilli, @Andreas Stuhlmüller, @Andrew Dai, @Andrew La, @Andrew Lampinen, @Andy Zou, @Angela Jiang, @Angelica Chen, @Anh Vuong, @Animesh Gupta, @Anna Gottardi, @Antonio Norelli, @Anu Venkatesh, @Arash Gholamidavoodi, @Arfa Tabassum, @Arul Menezes, @Arun Kirubarajan, @Asher Mullokandov, @Ashish Sabharwal, @Austin Herrick, @Avia Efrat, @Aykut Erdem, @Ayla Karakaş, @B. Ryan Roberts, @Bao Sheng Loe, @Barret Zoph, @Bartłomiej Bojanowski, @Batuhan Özyurt, @Behnam Hedayatnia, @Behnam Neyshabur, @Benjamin Inden, @Benno Stein, @Berk Ekmekci, @Bill Yuchen Lin, @Blake Howald, @Cameron Diao, @Cameron Dour, @Catherine Stinson, @Cedrick Argueta, @César Ferri Ramírez, @Chandan Singh, @Charles Rathkopf, @Chenlin Meng, @Chitta Baral, @Chiyu Wu, @Chris Callison-Burch, @Chris Waites, @Christian Voigt, @Christopher D. Manning, @Christopher Potts, @Cindy Ramirez, @Clara E. Rivera, @Clemencia Siro, @Colin Raffel, @Courtney Ashcraft, @Cristina Garbacea, @Damien Sileo, @Dan Garrette, @Dan Hendrycks, @Dan Kilman, @Dan Roth, @Daniel Freeman, @Daniel Khashabi, @Daniel Levy, @Daniel Moseguí González, @Danielle Perszyk, @Danny Hernandez, @Danqi Chen, @Daphne Ippolito, @Dar Gilboa , @David Dohan, @David Drakard, @David Jurgens, @Debajyoti Datta, @Deep Ganguli, @Denis Emelin, @Denis Kleyko, @Deniz Yuret, @Derek Chen, @Derek Tam, @Dieuwke Hupkes, @Diganta Misra, @Dilyar Buzan, @Dimitri Coelho Mollo, @Diyi Yang, @Dong-Ho Lee, @Ekaterina Shutova, @Ekin Dogus Cubuk, @Elad Segal, @Eleanor Hagerman, @Elizabeth Barnes, @Elizabeth Donoway, @Ellie Pavlick, @Emanuele Rodola, @Emma Lam, @Eric Chu, @Eric Tang, @Erkut Erdem, @Ernie Chang, @Ethan A. Chi, @Ethan Dyer, @Ethan Jerzak, @Ethan Kim, @Eunice Engefu Manyasi, @Evgenii Zheltonozhskii, @Fanyue Xia, @Fatemeh Siar, @Fernando Martínez-Plumed, @Francesca Happé, @Francois Chollet, @Frieda Rong, @Gaurav Mishra, @Genta Indra Winata, @Gerard de Melo, @Germán Kruszewski, @Giambattista Parascandolo, @Giorgio Mariani, @Gloria Wang, @Gonzalo Jaimovitch-López, @Gregor Betz, @Guy Gur-Ari, @Hana Galijasevic, @Hannah Kim, @Hannah Rashkin, @Hannaneh Hajishirzi, @Harsh Mehta, @Hayden Bogar, @Henry Shevlin, @Hinrich Schütze, @Hiromu Yakura, @Hongming Zhang, @Hugh Mee Wong, @Ian Ng, @Isaac Noble, @Jaap Jumelet, @Jack Geissinger, @Jackson Kernion, @Jacob Hilton, @Jaehoon Lee, @Jaime Fernández Fisac, @James B. Simon, @James Koppel, @James Zheng, @James Zou, @Jan Kocoń, @Jana Thompson, @Jared Kaplan, @Jarema Radom, @Jascha Sohl-Dickstein, @Jason Phang, @Jason Wei, @Jason Yosinski, @Jekaterina Novikova, @Jelle Bosscher, @Jennifer Marsh, @Jeremy Kim, @Jeroen Taal, @Jesse Engel, @Jesujoba Alabi, @Jiacheng Xu, @Jiaming Song, @Jillian Tang, @Joan Waweru, @John Burden, @John Miller, @John U. Balis, @Jonathan Berant, @Jörg Frohberg, @Jos Rozen, @Jose Hernandez-Orallo, @Joseph Boudeman, @Joseph Jones, @Joshua B. Tenenbaum, @Joshua S. Rule, @Joyce Chua, @Kamil Kanclerz, @Karen Livescu, @Karl Krauth, @Karthik Gopalakrishnan, @Katerina Ignatyeva, @Katja Markert, @Kaustubh D. Dhole, @Kevin Gimpel, @Kevin Omondi, @Kory Mathewson, @Kristen Chiafullo, @Ksenia Shkaruta, @Kumar Shridhar, @Kyle McDonell, @Kyle Richardson, @Laria Reynolds, @Leo Gao, @Li Zhang, @Liam Dugan, @Lianhui Qin, @Lidia Contreras-Ochando, @Louis-Philippe Morency, @Luca Moschella, @Lucas Lam, @Lucy Noble, @Ludwig Schmidt, @Luheng He, @Luis Oliveros Colón, @Luke Metz, @Lütfi Kerem Şenel, @Maarten Bosma, @Maarten Sap, @Maartje ter Hoeve, @Maheen Farooqi, @Manaal Faruqui, @Mantas Mazeika, @Marco Baturan, @Marco Marelli, @Marco Maru, @Maria Jose Ramírez Quintana, @Marie Tolkiehn, @Mario Giulianelli, @Martha Lewis, @Martin Potthast, @Matthew L. Leavitt, @Matthias Hagen, @Mátyás Schubert, @Medina Orduna Baitemirova, @Melody Arnaud, @Melvin McElrath, @Michael A. Yee, @Michael Cohen, @Michael Gu, @Michael Ivanitskiy, @Michael Starritt, @Michael Strube, @Michał Swędrowski, @Michele Bevilacqua, @Michihiro Yasunaga, @Mihir Kale, @Mike Cain, @Mimee Xu, @Mirac Suzgun, @Mo Tiwari, @Mohit Bansal, @Moin Aminnaseri, @Mor Geva, @Mozhdeh Gheini, @Mukund Varma T, @Nanyun Peng, @Nathan Chi, @Nayeon Lee, @Neta Gur-Ari Krakover, @Nicholas Cameron, @Nicholas Roberts, @Nick Doiron, @Nikita Nangia, @Niklas Deckers, @Niklas Muennighoff, @Nitish Shirish Keskar, @Niveditha S. Iyer, @Noah Constant, @Noah Fiedel, @Nuan Wen, @Oliver Zhang, @Omar Agha, @Omar Elbaghdadi, @Omer Levy, @Owain Evans, @Pablo Antonio Moreno Casares, @Parth Doshi, @Pascale Fung, @Paul Pu Liang, @Paul Vicol, @Pegah Alipoormolabashi, @Peiyuan Liao, @Percy Liang, @Peter Chang, @Peter Eckersley, @Phu Mon Htut, @Pinyu Hwang, @Piotr Miłkowski, @Piyush Patil, @Pouya Pezeshkpour, @Priti Oli, @Qiaozhu Mei, @Qing Lyu, @Qinlang Chen, @Rabin Banjade, @Rachel Etta Rudolph, @Raefer Gabriel, @Rahel Habacker, @Ramón Risco Delgado, @Raphaël Millière, @Rhythm Garg, @Richard Barnes, @Rif A. Saurous, @Riku Arakawa, @Robbe Raymaekers, @Robert Frank, @Rohan Sikand, @Roman Novak, @Roman Sitelew, @Ronan LeBras, @Rosanne Liu, @Rowan Jacobs, @Rui Zhang, @Ruslan Salakhutdinov, @Ryan Chi, @Ryan Lee, @Ryan Stovall, @Ryan Teehan, @Rylan Yang, @Sahib Singh, @Saif M. Mohammad, @Sajant Anand, @Sam Dillavou, @Sam Shleifer, @Sam Wiseman, @Samuel Gruetter, @Samuel R. Bowman, @Samuel S. Schoenholz, @Sanghyun Han, @Sanjeev Kwatra, @Sarah A. Rous, @Sarik Ghazarian, @Sayan Ghosh, @Sean Casey, @Sebastian Bischoff, @Sebastian Gehrmann, @Sebastian Schuster, @Sepideh Sadeghi, @Shadi Hamdan, @Sharon Zhou, @Shashank Srivastava, @Sherry Shi, @Shikhar Singh, @Shima Asaadi, @Shixiang Shane Gu, @Shubh Pachchigar, @Shubham Toshniwal, @Shyam Upadhyay, @Shyamolima (Shammie)Debnath, @Siamak Shakeri, @Simon Thormeyer, @Simone Melzi, @Siva Reddy, @Sneha Priscilla Makini, @Soo-Hwan Lee, @Spencer Torene, @Sriharsha Hatwar, @Stanislas Dehaene, @Stefan Divic, @Stefano Ermon, @Stella Biderman, @Stephanie Lin, @Stephen Prasad, @Steven T. Piantadosi, @Stuart M. Shieber, @Summer Misherghi, @Svetlana Kiritchenko, @Swaroop Mishra, @Tal Linzen, @Tal Schuster, @Tao Li, @Tao Yu, @Tariq Ali, @Tatsu Hashimoto, @Te-Lin Wu, @Théo Desbordes, @Theodore Rothschild, @Thomas Phan, @Tianle Wang, @Tiberius Nkinyili, @Timo Schick, @Timofei Kornev, @Timothy Telleen-Lawton, @Titus Tunduny, @Tobias Gerstenberg, @Trenton Chang, @Trishala Neeraj, @Tushar Khot, @Tyler Shultz, @Uri Shaham, @Vedant Misra, @Vera Demberg, @Victoria Nyamai, @Vikas Raunak, @Vinay Ramasesh, @Vinay Uday Prabhu, @Vishakh Padmakumar, @Vivek Srikumar, @William Fedus, @William Saunders, @William Zhang, @Wout Vossen, @Xiang Ren, @Xiaoyu Tong, @Xinran Zhao, @Xinyi Wu, @Xudong Shen, @Yadollah Yaghoobzadeh, @Yair Lakretz, @Yangqiu Song, @Yasaman Bahri, @Yejin Choi, @Yichi Yang, @Yiding Hao, @Yifu Chen, @Yonatan Belinkov, @Yu Hou, @Yufang Hou, @Yuntao Bai, @Zachary Seid, @Zhuoye Zhao, @Zijian Wang, @Zijie J. Wang, @Zirui Wang, @Ziyi Wu
