Selected Projects

Critic Based Empathetic Actor Updates in Sequential Social Dilemmas.
Multi-Agent RL, Social Dilemma

• Background : In navigating complex social dilemmas, human groups often seek avenues for cooperation. Traditional behavioral economics models are limited to explaining cooperative behavior in simple, static scenarios like matrix games and fail to consider temporal dynamics. Recent advancements in multi-agent reinforcement learning (MARL) offer a more comprehensive approach. Research indicates that many people display a preference against inequity, which influences their behavior in social dilemmas, promoting cooperation and negative feelings if they are the only defectors.
  

• Research Question :
  1. How does the expectation of future rewards that other agents receive impact empathetic social interactions? Specifically:
     
     • Do empathetic agents become even more unselfish since they expect others to behave similarly?
       
     • How does this translate to human interactions, particularly with unknown individuals?
       

• Contribution : Our project aims to promote cooperation among agents without relying on access to rewards. We focus on intrinsic motivation, challenging the assumption that agents need to know others’ rewards to cooperate.
  

• Results : Our findings suggest that agents can learn to cooperate based on intrinsic motivations alone. Simple Inequity Aversion (SIA) demonstrates that agents may develop cooperative behaviors without needing to predict or access other agents’ rewards, thereby promoting more robust cooperation in social dilemmas.

Why do Researchers Prefer to use PyTorch Over TensorFlow?
Data Science, Machine Learning, Software Analytics

• Background : Deep learning has achieved significant success across various domains, leading to the development of popular frameworks like PyTorch and TensorFlow, released by Meta and Google in 2015 and 2016, respectively. Despite being released later, PyTorch has gained more traction among researchers. This project aims to provide an in-depth analysis of both frameworks to help software developers understand the important features that researchers need in future libraries.
  
• Research Question :
  1. Which framework offers greater ease of use for implementing deep learning models through their respective APIs?
     
  2. How does the prevalence of bugs in each framework influence their respective popularity over time?
     
• Contribution :
  1. API Analysis: We thoroughly investigate the APIs in both frameworks, analyzing client code and documentation using quantitative metrics for a comprehensive understanding.
     
  2. Bug Evolution: We examine the evolution of bugs in both frameworks to understand their respective popularity over time.
     
• Results : Our study reveals that:
  
  1. Implementing deep learning models using PyTorch APIs requires less effort compared to TensorFlow.
     
  2. PyTorch documentation is more detailed and its research community grows faster, leading to quicker issue resolution compared to TensorFlow.
     
     
     

Accelerating Disk-Based Approximate Nearest Neighbor Search via Inverted Indexing.
Machine Learning, Search Optimization, NLP

• Background : The field of approximate nearest neighbor searches (ANNs) has seen significant progress, particularly in developing disk-resident indices to manage massive datasets. Microsoft’s SPANN algorithm stands out for its hybrid use of memory and disk storage, enabling efficient retrieval of nearest neighbors for given queries. However, recent advancements have primarily focused on improving index-building techniques, with little attention paid to optimizing large-scale query batches or supporting filtered searches with labeled and unstructured data.
  
• Research Question : How can the performance of disk-based approximate nearest neighbor searches be enhanced for large-scale query batches and filtered searches, while maintaining high recall and graph quality?
  
• Contribution : In this project, we have two key enhancements to the SPANN model. First, an inverted-index-based, out-of-order query execution engine is proposed, which significantly accelerates current disk-based ANNs. Second, the SPANN model is extended to support the addition of filters in the search process, improving the handling of labeled and unstructured data.
  
• Results : Experiments demonstrate that the proposed query execution engine can outperform the state-of-the-art SPANN model by up to 6x while maintaining the same level of recall. Qualitative analysis confirms that the graph quality is preserved even with the support for filters, ensuring the integrity of the search results.