crossmodal

This project focuses on social media multimodal hate detection, addressing challenges such as image–text semantic inconsistency, implicit references, and sarcastic/ironic expressions. We develop a CLIP + LLMs based framework for multimodal representation learning and semantic alignment. Prompt Engineering (PE) optimizes LLM output, while multi-LLM

PEANUT (Prompt-Enhanced Ablation with Optical Flow-Based Neural Unit) designed to enhance video restoration by combining spatial and temporal consistency with clarity optimization. The core innovation lies in Prompt-Guided Mask Self-Generation and leveraging optical flow-based neural units to generate high-fidelity video sequences

Emotionally paired Music and Image Dataset (EMID) is a novel dataset designed for the emotional matching of music and images.

Public page for open vocabulary detection contest 2023

A powerful multimodal document search engine that converts PDF documents into searchable vector embeddings using OpenAI's CLIP model. Enables cross-modal search across text and images with natural language queries through a modern Streamlit interface.

Qbind is a new language designed to allow humans to intuitively process and embed data across different modalities. By using qbind, users can define relationships between various modalities by specifying simple operations and transformations on the data.

This patent-pending work pioneers a cross-modal knowledge distillation framework optimized for edge AI deployment, integrating modality-specific compression and lightweight feature alignment techniques.

Code for Bachelor Thesis "Unveiling Hidden Features: Multimodal Integration Using Cross-Modal Variational Autoencoders for the Identification of Stratification in ABIDE"