Ayaan Haque

Hi! I'm Ayaan Haque, a 19 y/o at UC Berkeley studying Electrical Engineering and Computer Science (EECS).

My research interests are in deep learning and computer vision. I'm currently advised by Prof. Angjoo Kanazawa at BAIR on NeRFs and generative 3D vision research. This summer, I'll be working at Luma AI working on 3D foundation models.

In the past, I've worked on self-supervised and unsupervised representation learning. Last summer, I interned at Samsung SDSA working on 3D contrastive learning. I got my research career jumpstarted with the Wang Group at Stanford’s RSL, where I addressed clinical imaging tasks using semi-supervised, self-supervised, and multi-task learning. Now at Berkeley, I'm a part of Machine Learning @ Berkeley and the Nerfstudio team, a large-scale, open-sourced NeRF development project.

In a past life, I was a builder and hacker (I'm a MLH Top-50 Hacker!), and now I'm exploring deep-tech startups. Most importantly, my projects are centered around making an impact in my communities, whether it be locally or all the way back in Bangladesh. Other than that, I enjoy writing, watching/playing sports, eating out with friends, and just having a good time. My ongoing goal and dream:

Twitter  /  Email  /  Resume  /  Github  /  Google Scholar  /  Medium  /  LinkedIn

• [May 2023] Joining Luma AI, a Series A startup building the future of 3D!
• [Mar 2023] Released new pre-print Instruct-NeRF2NeRF!
• [Feb 2023] Attending AAAI 2023 in Washington D.C!
• [Dec 2022] Joined the Kanazawa AI Lab at BAIR!
• [Nov 2022] SSL-MeshCNN accepted to AAAI 2023!
• [Sep 2022] Joined Machine Learning @ Berkeley!
• [Aug 2022] Released new preprint with Samsung on contrastive learning for mesh segmentation
• [June 2022] Joining Samsung SDSA as a Research Intern
• [April 2022] Committed to UC Berkeley to study EECS! (First update)

Since I've worked on a wide variety of topics (and am still exploring new topics), I've split my publications into currently relevant works and previous works. Only relevant works are listed below. For a full list (and chronological list) of my papers, visit my Google Scholar.

We propose a method for editing NeRF scenes with text-instructions. Given a NeRF of a scene and the collection of images used to reconstruct it, our method uses an image-conditioned diffusion model (InstructPix2Pix) to iteratively edit the input images while optimizing the underlying scene, resulting in an optimized 3D scene that respects the edit instruction.

We introduce self-supervised MeshCNN, or SSL-MeshCNN, a novel mesh-specialized contrastive learning method to perform downstream segmentation with limited-labeled data. We create an augmentation policy tailored for meshes, enabling the network to learn efficient visual representations through contrastive pre-training.

We propose SDCNL to address the unexplored problem of classifying between depression and more severe suicidal tendencies using web-scraped data. Our method introduces a novel label correction method to remove inherent noise in web-scraped data using unsupervised learning combined with a deep-learning classifier based on pre-trained transformers.

We introduce MultiMix, a joint semi-supervised classification and segmentation model employing a confidence-based augmentation strategy for semi-supervised classification along with a novel saliency bridge module that guides segmentation and provides explainability for the joint tasks.

We propose EC-GAN, which combines a Generative Adversarial Network with a classifier to leverage artifical GAN generations to increase the size of restricted, fully-supervised datasets using semi-supervised algorithms. Mentored by Microsoft Postdoc and Princeton University PhD Jordan T. Ash.

We propose Noise2Quality (N2Q), a novel, self-supervised IQA model which predicts SSIM Image Quality maps from low-dose CT. We propose a self-supervised regularization task of dose-level estimation creating a multi-tasking framework to improve performance.

We introduce SSWL-IDN, a novel self-supervised CT denoising window-level prediction surrogate task. Our method is task-relevant and related to the downstream task, yielding improved performance over recent methods.

We expand upon MultiMix (in ISBI 2021). Our extended manuscript contains a detailed explanation of the methods, saliency map visualizations from multiple datasets, and quantitative (performance metrics tables) and qualitative (mask predictions, Bland Altman plots, ROC curves, consistency plots).

We propose FCE-NN, a novel method of modeling robotic launching of non-rigid objects using neural networks which are trained with supplemental simulated data, generated from algorithmic force coefficient estimation.

I've held both research and SWE roles at various companies, startups, and universities.

• Neural rendering startup building the future of 3D content creation, raised $20 million Series A
• Building 3D foundational models!

• Working on NeRFs and diffusion models

• Proposed "SSL-MeshCNN", a novel self-supervised algorithm for segmenting non-uniform, irregular 3D meshes
• Introduced new SimCLR-inspired stochastic augmentation policy for mesh-specialized contrastive learning
• Matched accuracy of fully-supervised training (90.50%) with just 67% of labels on benchmark datasets
• Wrote paper accepted to AAAI 2023, available on ArXiv

• Worked on learning from limited labeled data for clinical imaging tasks using unsupervised, self-supervised, and semi-supervised techniques
• Developed research skills by running hundreds of experiments, writing papers, preparing supplementals, and writing rebuttals

• Accelerator providing early-stage startups with software and human capital instead of direct funding (closed as of late 2021)
• Developing a Waitlist API which is to be sold to porfolio companies in the program, where companies can establish waitlists for their products to build a market
• Using React.js, MongoDB, Flask and other web dev/backend tech, integrating Stripe payment features and referral features, writing documentation

Outside of research, I enjoy building practical applications in both competitive and casual formats.

• Team Captain of 5 total members (shoutout Viraaj, Adithya, Ishaan, and Sajiv)
• Created numerous projects (listed in Projects section)
• 🏆 33x Award Winner, 9x First Place, 22x Top 3, $10,000+ in earnings
• Chosen for MLH Top 50 Hackers Class of 2021, one of five high schoolers

• FRC Robotics Software Team, ML-Specialist (FTC Captain 9th Grade)
• Worked with AI/ML for in-game object detection and using predictive models for shot selection, work on shooter trajectory modeling, write documentation
• 🏆 2021 Skills Competition Finalist in Carbon Group 🏆 2021 Engineering Excellence Award 🏆 CalGames 2019 Finalist 🏆 ChezyChamps 2019 Semi-Finalist

I've just listed a few of my favorite projects, and the remaining are available on my Github. On Github, I have 500+ commits and ~300 stars across all my repositories. Check out this cool commit graph, and check this out for my Github stats.

Contribute (in small parts) to large-scale open-source project. Helped implement research methods (Instruct-NeRF2NeRF, CLIP-based NeRF) into main repository.

Stack: Python, PyTorch

SuiSense is a progressive web application that uses Artificial Intelligence (AI) and Natural Language Processing (NLP) to distinguish between depressive and suicidal phrases and help concerned friends and family determine whether their struggling loved one is on the path to suicide. SuiSense uses an implementation of SDCNL.

🏆 4th Place Congressional App Challenge 2020 🏆 2nd Place @ GeomHacks 2020 🏆 HM @ MLH Summer League SHDH 2020

Stack: Python, HTML, CSS, JavaScript, Tensorflow, PyTorch, BERT, Flask, PythonAnywhere, Pandas, Sci-Kit Learn

This mobile, on-the-go system is designed for clinics in Bangladesh to screen patients for Diabetic Retinopathy (DR) using a smartphone camera with a retinal attachment. The purpose of this rig is to allow precise positioning of the smartphone to any patient's left and right eye such that the images can be efficiently fed into Drishti's AI algorithms for DR diagnosis. The system is completely adjustable for all head sizes. It is made of readily available components that can be purchased at many local hardware stores, and is designed for low-cost fabrication. All assembly tools are common household tools or easily purchasable/rentable from a local hardware store. The 3D-printed components can be printed on low-end machines and with cheap PLA filament. We designed this system to be completely collapsable, such that it can fit into a standard size backpack.

Stack: SolidWorks, Hardware Materials

Tickbird is an advanced Swift mobile app based on the TesseractOCR neural network framework allowing visually impaired patients to aurally understand their prescriptions or the labels on their pill bottles in order to gain independence and avoid the prospect of lethal miscommunication regarding necessary medicines from their doctors. Moreover, the app's smart profiling feature not only finds the nearest pharmacy containing the user's prescription, but it also uses AI/ML algorithms to detect and set notifications for the times the user has to take or refill their medicine.

🏆 2x Award Winner @ OmniHacks 2019 🏆 App Store April 2020, 1000+ Impresions

Stack: Swift, Xcode, IOS, Firebase, TesseractOCR, Ruby

I write on Medium (semi-regularly) to share my thoughts with the world. Here are a few of my favorite medium articles that I have written.

A response to an article discussing our purpose in this world combined with a discussion of my own purpose

This article is a reponse to my friend's article, where he discusses critiques of our Saratoga society. In my article, I respond to his ideas and then share my own story of finding my purpose in life.

Coding, winning prizes, and proving ourselves

Authored by Ayaan Haque, Adithya Peruvemba, Viraaj Reddi, Sajiv Shah, and Ishaan Bhandari

This article travels through the journey of my team, Haleakala Hacksquad, and how we became great hackers.

With my technical skills, I love contributing to my community and learning the stories of those I support. Whether these are my local communities or my home country Bangladesh, I always build strong relationships with those I serve.

Drishti is an organization with an AI algorithm that screens Bangladeshi patients for diabetic retinopathy (DR). Our mission is to provide free, accessible early screening to areas where DR specialists are not available. With academic, clinical, and organizational support, we are able to widen the reach of our service. We are partnered with BRAC University and have published a validation study on the algorithm, achieving high accuracy on Bangladeshi eyes. In addition, we have developed a novel smartphone retinal camera system (patent application in process) which will be integrated into our clinics.

Jaago Robotics is an organization partnered with the Jaago Foundation to teach robotics and coding to students at the Jaago School (a tuition free institute). As a branch of the Jaago Foundation, we use the Lego Mindstorms EV3 Robotics kit to help young students learn introductory robotics concepts, providing new opportunities for bright students. We go once every year and have gone twice so far (in the summer of 2018 and winter of 2019) and have taught over 20+ students.

I founded Jaago Robotics with my brother the summer of 2018, as a rising freshman. We have achieved so much with the students, most notably when the students presented their work to sponsors from Levi Strauss to receive funding for their school. We plan to return in December of 2021 to work with even more students.

Check out my social media!

Youtube  /  Twitter  /  Spotify

Website template from Jon Barron