Moodify

Moodify is an AI-powered music recommendation web application that intelligently detects your facial expressions and curates a customized music playlist that perfectly matches your current mood.

💻 Technologies Used

Frontend:

• React (Vite)
• Framer Motion (Animations)
• Zustand (State Management)
• React Router (Navigation)
• MediaPipe Tasks Vision (Real-time Emotion Detection)
• Google OAuth (Authentication)

Backend:

• Node.js & Express
• MongoDB (Mongoose)
• ImageKit (Cloud storage for audio and cover art)
• Node-ID3 & Music-Metadata (Audio metadata extraction)
• JWT & Bcryptjs (Security and Authentication)

---

✨ Features: What can users do?

• Real-Time Emotion Detection: Enable your webcam to let the app detect your current facial expression (Happy, Sad, Neutral, Angry, etc.).
• Mood-Based Playlists: Automatically fetch and play curated songs that match your detected mood.
• Custom Music Player: Fully functional audio player with play, pause, seek, volume control, mute, and skip functionalities.
• Upload Songs: Admins/Users can upload MP3 files. The app automatically extracts ID3 metadata (title, artist) and cover art.
• Authentication: Secure login and registration using Email/Password or Google OAuth.
• Theme Switching: Seamlessly toggle between Dark and Light modes (persisted via local storage).

---

⌨️ Keyboard Shortcuts

Enhance your music playback experience with these handy shortcuts (Note: Some shortcuts are planned for future updates):

• Spacebar : Play / Pause
• M : Mute / Unmute
• Right Arrow : Seek Forward / Skip Next
• Left Arrow : Seek Backward / Skip Previous

---

🛠️ The Process: How I Built It

1. Initialization: Started by setting up a monorepo-style structure with a Vite-powered React frontend and an Express Node.js backend.
2. AI Integration: Integrated Google's MediaPipe Tasks Vision in the frontend to process webcam feeds and predict user emotions in real time without server latency.
3. Backend & Storage Architecture: Designed a MongoDB schema for users and songs. Integrated ImageKit to handle the storage of audio files and extracted album art.
4. Metadata Extraction: Used node-id3 and music-metadata on the backend to automatically parse uploaded MP3 files, extracting song duration, title, artist, and embedded cover images to streamline the upload process.
5. State Management & UI: Used Zustand to manage complex global states (like audio playback context and authentication). Built a custom audio hook (useAudioPlayer) and styled the interface with beautiful animations using Framer Motion.

---

🧠 What I Learned

• Client-Side Machine Learning: How to efficiently integrate and run computer vision models (MediaPipe) directly in the browser for real-time tracking.
• Audio Processing: How to handle binary audio buffers in Node.js, upload them securely, and parse complex ID3 metadata.
• State Management: Deepened my understanding of managing persistent and global state without prop drilling using Zustand.
• Cloud Integration: Efficiently managing file streams and separating concerns using ImageKit for media delivery.

---

🚀 How It Could Be Improved

• Enhanced Keyboard Shortcuts: Implement global event listeners for seamless keyboard media controls.
• Mobile Responsiveness: Optimize the webcam layout and music player interface for smaller screens.
• Personalized Playlists: Allow users to like songs, create custom playlists, and view their mood history.
• Lyrics Syncing: Integrate an API to display real-time lyrics as the song plays.
• Advanced AI Models: Fine-tune the emotion detection model for higher accuracy across different lighting conditions and demographics.

---

⚙️ How to Run the Project Locally

Prerequisites

• Node.js installed
• MongoDB URI
• ImageKit Account (Public Key, Private Key, URL Endpoint)
• Google OAuth Client ID

1. Clone the repository

git clone https://github.com/your-username/moodify.git
cd moodify

2. Backend Setup

cd Backend
npm install

Create a .env file in the Backend directory:

PORT=3000
MONGO_URI=your_mongodb_connection_string
JWT_SECRET=your_secret_key
IMAGEKIT_PUBLIC_KEY=your_public_key
IMAGEKIT_PRIVATE_KEY=your_private_key
IMAGEKIT_URL_ENDPOINT=your_url_endpoint

Start the backend server:

npm run dev

3. Frontend Setup

Open a new terminal and navigate to the frontend:

cd Frontend
npm install

Create a .env file in the Frontend directory:

VITE_GOOGLE_CLIENT_ID=your_google_client_id
VITE_API_URL=http://localhost:3000

Start the frontend development server:

npm run dev

---

🎥 Live Project Demo

Watch the project in action below:

https://github.com/anaghroy/Moodify/releases/download/v1.0.0/output_progressive_3c9457fd-c96a-4bb2-9cae-16653c80aff1.gif