Di Lu (卢迪)

Graduate Student

Blender Lab

Rensselaer Polytechnic Institute

Troy, NY 12180

Office: Room 2148, Winslow

Email: lud2 at rpi dot edu

[CV] [Github] [Resource]

I'm a fourth year Ph.D. student in Electrical, Computer and System Engineering department at Rensselaer Polytechnic Institute. I am working with Prof. Heng Ji at Blender Lab. Prior to my study at RPI, I received my B.S. degree in Physics from University of Science and Technology of China, and M.S. degree in EE from Boston University.

My research interests include Natural Language Processing, Computer Vision, and Machine Learning, with a focus on multimodal information extraction, transfer and generation.


  • Our paper on Multimodal Name Tagging in social media is accepted in ACL2018!
  • Our demo paper on Cross-lingual Entity Extraction, Linking and Localization is accepted in NAACL2018!


  • Entity-aware Image Caption Generation
    Di Lu, Spencer Whitehead, Lifu Huang, Heng Ji, Shih-Fu Chang

Selected Publications

  • Visual Attention Model for Name Tagging in Multimodal Social Media
    Di Lu, Leonardo Neves, Vitor Carvalho, Ning Zhang, Heng Ji
    ACL 2018

  • ELISA-EDL: A Cross-lingual Entity Extraction, Linking and Localization System
    Boliang Zhang, Ying Lin, Xiaoman Pan, Di Lu, Jonathan May, Kevin Knight and Heng Ji
    NAACL-HLT 2018 (Demo Paper)
    [paper][ie demo][map demo]

  • Embracing Non-Traditional Linguistic Resources for Low-resource Language Name Tagging
    Boliang Zhang, Di Lu, Xiaoman Pan, Ying Lin, Halidanmu Abudukelimu, Heng Ji, Kevin Knight
    IJCNLP 2017

  • Team ELISA System for DARPA LORELEI Speech Evaluation 2016
    Pavlos Papadopoulos, Ruchir Travadi, Colin Vaz, Nikolaos Malandrakis, Ulf Hermjakob, Nima Pourdamghani, Michael Pust, Boliang Zhang, Xiaoman Pan, Di Lu, Ying Lin, Ondrej Glembek, Murali Karthick B, Martin Karafiat, Lukas Burget, Mark Hasegawa-Johnson, Heng Ji, Jonathan May, Kevin Knight and Shrikanth Narayanan
    In proceedings of the Interspeech2017

  • Incident-Driven Machine Translation and Name Tagging for Low-resource Languages
    Ulf Hermjakob, Qiang Li, Daniel Marcu, Jonathan May, Sebastian J. Mielke, Nima Pourdamghani, Michael Pust, Xing Shi, Kevin Knight, Tomer Levinboim, Kenton Murray, David Chiang, Boliang Zhang, Xiaoman Pan, Di Lu, Ying Lin and Heng Ji
    Machine Translation 2017

  • A Multi-media Approach to Cross-lingual Entity Knowledge Transfer
    Di Lu, Xiaoman Pan, Nima Pourdamghani, Shih-Fu Chang, Heng Ji and Kevin Knight
    ACL 2016 (oral)

  • Cross-media Event Extraction and Recommendation
    Di Lu, Clare Voss, Fangbo Tao, Xiang Ren, Rachel Guan, Rostyslav Korolov, Tongtao Zhang, Dongang Wang, Hongzhi Li, Taylor Cassidy, Heng Ji, Shih-fu Chang, Jiawei Han, William Wallace, James Hendler, Mei Si and Lance Kaplan
    NAACL-HLT 2016 (Demo Paper)
    [paper] [demo]

  • On Predicting Social Unrest Using Social Media
    Rostyslav Korolov, Di Lu, Jingjing Wang, Guangyu Zhou, Claire Bonial, Clare Voss, Lance Kaplan, William Wallace, Jiawei Han and Heng Ji
[complete list]

Work Experience

  • Research Intern

    Sep-Dec 2017|Venice, CA, USA

    Snap Inc.

Teaching Experience

  • ENGR 2500, Electronic Instrumentation, Fall 2014
    Rensselaer Polytechnic Institute, Troy, NY

  • ECSE 2050, Introduction to Electronics, Fall 2014
    Rensselaer Polytechnic Institute, Troy, NY


Ph.D. in Computer and System Engineering, Rensselaer Polytechnic Institute, Fall 2014-Present

M.S. in Electrical Engineering, Boston University, Fall 2010-May 2012

B.S. in Physics, University of Science and Technology of China, Fall 2004-July 2008

Earlier Research

Check some of my earlier fun projects


Music, Instrument and Movie