pianorhythm-ssr

Core Business Logic (pianorhythm_core)

The pianorhythm_core is the heart of PianoRhythm, implemented in Rust and compiled to WebAssembly for high-performance audio processing, 3D rendering, and real-time communication.

Overview

The core engine is structured as a modular Rust workspace with the following main components:

pianorhythm_core/
├── core/           # Main application logic and state management
├── synth/          # Audio synthesis engine
├── bevy_renderer/  # 3D rendering with Bevy Engine
├── shared/         # Shared utilities and types
├── proto/          # Protocol Buffer definitions
└── desktop/        # Desktop-specific integrations

Core Module Architecture

State Management

The core uses a Redux-like pattern with reducers for state management:

pub struct AppState {
    pub client_state: ClientState,
    pub current_room_state: CurrentRoomState,
    pub rooms_list_state: RoomsState,
    pub audio_process_state: AudioProcessState,
    pub app_settings: AppSettings,
    pub app_environment: AppCommonEnvironment,
}

Key Reducers:

WebAssembly Interface

#[cfg(target_arch = "wasm32")]
#[wasm_bindgen]
pub fn create_synth(options: PianoRhythmSynthesizerDescriptor) -> Result<(), String> {
    unsafe {
        _ = SYNTH.set(PianoRhythmSynthesizer::new(
            options,
            Some(Box::new(|event| emit_to_note_buffer_engine(&event))),
            Some(Box::new(handle_synth_events)),
            Some(Box::new(handle_audio_channel_updates)),
        ));
    }
    Ok(())
}

WASM Exports:

Audio Synthesis Engine

Synthesizer Architecture

The audio engine is built on a custom synthesizer implementation:

pub struct PianoRhythmSynthesizer {
    synth: oxisynth::Synth,
    socket_users: HashMap<u32, PianoRhythmSocketUser>,
    client_socket_id: Option<u32>,
    soundfont_loaded: bool,
    // ... additional fields
}

Key Features:

Audio Processing Pipeline

impl PianoRhythmSynthesizer {
    pub fn process(&mut self, output: &mut [f32]) {
        let mut chunks = output.chunks_exact_mut(2);
        for chunk in &mut chunks {
            let (mut l, mut r) = self.read_next();
            self.equalize(&mut l);
            self.equalize(&mut r);
            chunk[0] = l;
            chunk[1] = r;
        }
    }
}

Processing Steps:

  1. Note Generation: MIDI events converted to audio samples
  2. Effects Processing: Apply reverb, chorus, and EQ
  3. Mixing: Combine multiple user channels
  4. Output: Stereo audio output to Web Audio API

Soundfont Management

pub fn load_soundfont(&mut self, soundfont_data: &[u8]) -> Result<(), String> {
    match self.synth.load_soundfont(soundfont_data) {
        Ok(_) => {
            self.soundfont_loaded = true;
            Ok(())
        }
        Err(e) => Err(format!("Failed to load soundfont: {}", e))
    }
}

Soundfont Features:

3D Rendering Engine (Bevy)

Bevy Integration

The 3D renderer uses Bevy Engine for high-performance graphics:

pub fn root_app() -> App {
    let mut app = App::new();
    
    app.add_plugins(DefaultPlugins.set(WindowPlugin {
        primary_window: Some(Window {
            canvas: Some("#bevy-canvas".to_string()),
            ..default()
        }),
        ..default()
    }))
    .add_plugins(CorePlugin)
    .add_plugins(PianoPlugin)
    .add_plugins(RoomPlugin);
    
    app
}

Rendering Features:

Component System

#[derive(Component, Reflect)]
pub struct PianoKey {
    pub key_id: u8,
    pub is_pressed: bool,
    pub velocity: f32,
}

#[derive(Component, Reflect)]
pub struct UserAvatar {
    pub socket_id: String,
    pub position: Vec3,
    pub color: Color,
}

Key Components:

Protocol Buffer Communication

Message Definitions

message AppStateActions {
  AppStateActions_Action action = 1;
  AudioSynthActions audioSynthAction = 2;
  string stringValue = 3;
  // ... additional fields
}

message AudioSynthActions {
  AudioSynthActions_Action action = 1;
  string socketId = 2;
  uint32 note = 3;
  uint32 velocity = 4;
  // ... additional fields
}

Message Types:

Serialization

pub fn send_app_action(&mut self, action: AppStateActions) {
    let bytes = action.write_to_bytes().unwrap();
    self.dispatch_action_bytes(&bytes);
}

Benefits:

MIDI Integration

MIDI Event Processing

pub trait HandleWebsocketMidiMessage {
    fn handle_websocket_midi_message(&mut self, data: &[u8]);
}

impl HandleWebsocketMidiMessage for AppState {
    fn handle_websocket_midi_message(&mut self, data: &[u8]) {
        if let Ok(midi_event) = MidiEvent::from_bytes(data) {
            self.process_midi_event(midi_event);
        }
    }
}

MIDI Features:

Memory Management

Rust Memory Safety

// Safe memory management with Rust's ownership system
pub struct NoteBufferEngine {
    buffer: Vec<MidiEvent>,
    capacity: usize,
}

impl NoteBufferEngine {
    pub fn add_event(&mut self, event: MidiEvent) {
        if self.buffer.len() < self.capacity {
            self.buffer.push(event);
        }
    }
}

Memory Features:

Error Handling

Robust Error Management

#[derive(Debug)]
pub enum CoreError {
    AudioInitializationFailed(String),
    SoundfontLoadError(String),
    WebSocketConnectionError(String),
    MidiDeviceError(String),
}

impl std::fmt::Display for CoreError {
    fn fmt(&self, f: &mut std::fmt::Formatter) -> std::fmt::Result {
        match self {
            CoreError::AudioInitializationFailed(msg) => {
                write!(f, "Audio initialization failed: {}", msg)
            }
            // ... other error types
        }
    }
}

Error Handling Strategy:

Performance Optimizations

Audio Performance

// Optimized audio processing with SIMD when available
#[cfg(target_feature = "simd128")]
fn process_audio_simd(input: &[f32], output: &mut [f32]) {
    // SIMD-optimized audio processing
}

Optimization Techniques:

WebAssembly Optimizations

// Optimized for WASM compilation
#[cfg(target_arch = "wasm32")]
#[wasm_bindgen]
pub struct WasmAudioProcessor {
    processor: AudioProcessor,
}

#[cfg(target_arch = "wasm32")]
#[wasm_bindgen]
impl WasmAudioProcessor {
    #[wasm_bindgen(constructor)]
    pub fn new() -> Self {
        Self {
            processor: AudioProcessor::new(),
        }
    }
}

WASM Features:

Build System

Cargo Configuration

[package]
name = "pianorhythm_core"
version = "0.1.0"
edition = "2021"

[dependencies]
wasm-bindgen = "0.2"
web-sys = "0.3"
js-sys = "0.3"
bevy = { version = "0.16", features = ["webgl2"] }
oxisynth = { path = "./synth/oxisynth" }

[lib]
crate-type = ["cdylib"]

Build Scripts:

Testing

Unit Tests

#[cfg(test)]
mod tests {
    use super::*;

    #[test]
    fn test_synthesizer_creation() {
        let options = PianoRhythmSynthesizerDescriptor::default();
        let synth = PianoRhythmSynthesizer::new(options, None, None, None);
        assert!(!synth.has_soundfont_loaded());
    }
}

Testing Strategy:

Next Steps