hetong.wang809[at]gmail.com

👋 Hi! I'm Hetong Wang.

My name is 王鹤童 in Chinese. Feel free to call me Hetong or Erika alternatively!

I play with data, structure, and model behavior concurrently to explore mechanistic interpretability. Recently, I have been particularly excited about the following questions:

Sparsity: How is information clustered and dispersed among the neurons of models? How do different training paradigms steer this distribution, and how do the circuits formed between neurons contribute to the generalisation performance?
Training Dynamics: How is knowledge acquired, and how does it evolve under varying data mixtures and learning curricula?

I believe that addressing these questions can lead to greater data and parameter efficiency while allowing us to reclaim more control from the black-box nature of neural networks.

About

I obtained my master in Artificial Intelligence at The Univerisity of Edinburgh, where I am fortunately advised by Prof. Edoardo Ponti and Prof. Pasquale Minervini. Before that, I obtained my bachelor in Computer Science at The University of Liverpool, with a research focus on Reinforcement Learning.

NEWS

May 15, 2024	🎉 Our paper Probing the Emergence of Cross-lingual Alignment during LLM Training is accepted at ACL 2024!
May 01, 2024	👩🏻‍🎓 I am actively looking for PhD position start by 25 spring/fall!
Jan 12, 2024	🗣️ I gave a talk on AI self-alignment at THUNLP&OpenBMB Lab. Check the recording here (in Manderin Chinese).
Dec 20, 2023	📍 I start my research internship on-site at Tsinghua University, Bejing.

PUBLICATIONS

ACL

Probing the Emergence of Cross-lingual Alignment during LLM Training

Hetong Wang, Pasquale Minervini, and Edoardo Ponti

In Findings of the 2024 Annual Meeting of the Association for Computational Linguistics: ACL, 2024

arXiv Bib Code

@inproceedings{wang2024probing,
  title = {Probing the Emergence of Cross-lingual Alignment during LLM Training},
  author = {Wang, Hetong and Minervini, Pasquale and Ponti, Edoardo},
  booktitle = {Findings of the 2024 Annual Meeting of the Association for Computational Linguistics: ACL},
  address = {Bangkok, Thailand},
  year = {2024},
  publisher = {The 62nd Annual Meeting of the Association for Computational Linguistics},
}

IEEE CASE

Accelerating Laboratory Automation Through Robot Skill Learning For Sample Scraping

Gabriella Pizzuto, Hetong Wang, Hatem Fakhruldeen, and 3 more authors

In 2024 IEEE International Conference on Automation Science and Engineering: CASE, 2024

Best Healthcare Automation Paper Award

arXiv Bib

@inproceedings{pizzuto2022accelerating,
  title = {Accelerating Laboratory Automation Through Robot Skill Learning For Sample Scraping},
  author = {Pizzuto, Gabriella and Wang, Hetong and Fakhruldeen, Hatem and Peng, Bei and Luck, Kevin S. and Cooper, Andrew I.},
  booktitle = {2024 IEEE International Conference on Automation Science and Engineering: CASE},
  eprint = {2209.14875},
  url = {https://arxiv.org/abs/2209.14875},
  year = {2024},
  archiveprefix = {arXiv},
  primaryclass = {cs.RO},
  publisher = {IEEE},
  note = {Best Healthcare Automation Paper Award}
}