Live Translator Setup Hub.

Run locally on the host PC first

Start the backend and open http://127.0.0.1:8000 on the same PC. This confirms Python, CUDA, the model, and microphone capture work before adding network complexity.

python -m venv .venv
.\.venv\Scripts\Activate.ps1
pip install -r requirements.txt
python main.py

Bind the backend to the LAN

Restart FastAPI on 0.0.0.0 so other devices can reach it, then test the health URL from the host PC and from another device.

$env:APP_HOST="0.0.0.0"
$env:APP_PORT="8000"
$env:WHISPER_MODEL="large-v3"
$env:WHISPER_DEVICE="cuda"
$env:WHISPER_COMPUTE_TYPE="float16"
python main.py

Open firewall ports

Allow TCP 8000 for direct health checks and TCP 8443 for the HTTPS website. Run this from an Administrator PowerShell window on the host PC.

New-NetFirewallRule -DisplayName "Live Translator 8000" -Direction Inbound -Action Allow -Protocol TCP -LocalPort 8000 -Profile Private
New-NetFirewallRule -DisplayName "Live Translator HTTPS 8443" -Direction Inbound -Action Allow -Protocol TCP -LocalPort 8443 -Profile Private

Add HTTPS for phone and laptop microphones

Browsers usually block microphone access on plain LAN HTTP. Use Caddy or another reverse proxy so the client opens https://192.168.0.20:8443 while FastAPI stays on http://127.0.0.1:8000.

https://192.168.0.20:8443 {
  tls ./translate-local.pem ./translate-local-key.pem
  reverse_proxy 127.0.0.1:8000
}

Open from the smartphone

Put the phone on the same Wi-Fi, open https://192.168.0.20:8443, accept or trust the certificate, press Start mic, and allow microphone permission. The phone sends audio; the RTX PC returns Korean transcript and English translation.

Deploy Firebase web config from GitHub variables

Add FIREBASE_API_KEY, FIREBASE_AUTH_DOMAIN, FIREBASE_PROJECT_ID, and FIREBASE_APP_ID as GitHub Actions variables. The workflow writes the public browser config during Pages deployment.

Start the protected backend on the RTX PC

Set-Location C:\Users\n\source\repos\Transcribe_translate
.\start-protected-https.ps1 `
  -FirebaseProjectId "your-firebase-project-id" `
  -AdminEmails "your-email@gmail.com" `
  -FirebaseCredentialsFile "C:\Users\n\secrets\firebase-service-account.json"

Forward the backend with HTTPS/WSS

For the first free test, run ngrok http https://127.0.0.1:8443 --host-header=rewrite and paste the HTTPS ngrok origin into the GitHub Pages backend box. Later, use DuckDNS + Caddy for a stable no-monthly-cost URL.

Approve users from the web admin panel

Unknown users become pending. Your admin account can refresh users, approve trusted emails, block abuse, and let approved users reconnect to use the GPU.

Keep the server path as the quality baseline

First test the S25 as a browser client against the RTX server. Record sample Korean phrases and save the output so the native app has a real comparison target.

Prototype native ASR before optimizing the NPU

Build the Android UI and microphone pipeline first. Test Android SpeechRecognizer.createOnDeviceSpeechRecognizer() or ML Kit GenAI Speech Recognition on the actual S25. If availability or Korean accuracy is not good enough, move to WhisperKit Android.

Add on-device Korean-to-English text translation

Once Korean text appears reliably, use ML Kit Translation for ko to en. Download models before travel and test in airplane mode.

Profile acceleration last

After the app works, test LiteRT/TFLite GPU or NPU delegates and Qualcomm AI Hub optimized models. Keep CPU and GPU fallbacks because Android NPU support is not plug-and-play for every model.

Check the GPU before choosing a model

Confirm the NVIDIA driver is visible, note total VRAM, and close heavy GPU apps before testing live translation.

nvidia-smi
python --version

Use the current faster-whisper CUDA path

Current faster-whisper/CTranslate2 GPU builds expect CUDA 12 libraries, including cuBLAS for CUDA 12 and cuDNN 9. If Python cannot load CUDA DLLs, fix the CUDA runtime before changing app code. For this stack, follow the faster-whisper README when it differs from general CTranslate2 installation notes.

python -m pip install --upgrade pip
pip install -r requirements.txt
python -c "import ctranslate2; print(ctranslate2.get_cuda_device_count())"

Start with the safe A1000 preset

This keeps latency reasonable while still showing the Korean transcript above the English translation. If the laptop is too slow, temporarily set SHOW_SOURCE_TEXT=false to skip the extra Korean transcription pass.

$env:WHISPER_MODEL="small"
$env:WHISPER_DEVICE="cuda"
$env:WHISPER_COMPUTE_TYPE="int8_float16"
$env:WHISPER_BEAM_SIZE="1"
$env:SHOW_SOURCE_TEXT="true"
python main.py

Upgrade to medium only after a stable test

If small is stable for 10 to 15 minutes and translation quality is not enough, try medium with the same quantized compute type. Increase WHISPER_BEAM_SIZE only if latency is still acceptable.

$env:WHISPER_MODEL="medium"
$env:WHISPER_DEVICE="cuda"
$env:WHISPER_COMPUTE_TYPE="int8_float16"
$env:WHISPER_BEAM_SIZE="1"
python main.py

Fallback when CUDA is not available

CPU mode is useful for installation checks and emergency offline use. It will usually feel slower than GPU mode, so use base first.

$env:WHISPER_MODEL="base"
$env:WHISPER_DEVICE="cpu"
$env:WHISPER_COMPUTE_TYPE="int8"
$env:WHISPER_CPU_THREADS="8"
python main.py

Prepare the RTX 4080 Super server

Install the NVIDIA driver, Python, Git, CUDA 12 runtime, and cuDNN 9. FFmpeg is optional for file conversion tests. The RTX 4080 Super has enough VRAM for large-v3, but keep other GPU-heavy apps closed during live translation.

nvidia-smi
python --version
git --version
ffmpeg -version

Install and test the translator backend

Run the Python backend on the server first. This command binds FastAPI to every network interface, so other computers can reach it after the firewall allows the port.

git clone https://github.com/buicongnguyen/ko-en-live-translator.git
cd ko-en-live-translator
python -m venv .venv
.\.venv\Scripts\Activate.ps1
pip install -r requirements.txt

$env:APP_HOST="0.0.0.0"
$env:APP_PORT="8000"
$env:WHISPER_MODEL="large-v3"
$env:WHISPER_DEVICE="cuda"
$env:WHISPER_COMPUTE_TYPE="float16"
$env:WHISPER_BEAM_SIZE="1"
python main.py

Find the server LAN address and open the firewall

Give the server a stable address in the router, or note its current IPv4 address. Allow TCP port 8000 for the backend and 443 or 8443 for HTTPS. Run the firewall commands in PowerShell as Administrator.

ipconfig
netsh advfirewall firewall add rule name="KO Translator Backend 8000" dir=in action=allow protocol=TCP localport=8000
netsh advfirewall firewall add rule name="KO Translator HTTPS 8443" dir=in action=allow protocol=TCP localport=8443

Add HTTPS for the laptop microphone

A laptop opening http://192.168.x.x:8000 is not the same as localhost, so browser microphone access may be blocked. Use HTTPS with a LAN hostname such as translate.local and reverse proxy to the FastAPI backend.

# Example Caddyfile on the server
https://translate.local:8443 {
  tls internal
  reverse_proxy 127.0.0.1:8000
}

Make the laptop resolve the server hostname

Use your router's local DNS if it supports custom hostnames. For a quick Windows test, add a hosts-file entry on the client laptop that points translate.local to the server IP.

# Run PowerShell as Administrator on the client laptop
Add-Content -Path "C:\Windows\System32\drivers\etc\hosts" -Value "192.168.1.50 translate.local"
ping translate.local

Open the website from the other laptop

On the client laptop, open the HTTPS address, allow microphone permission, and speak Korean near that laptop. The audio should stream to the RTX 4080 Super server and the browser should receive English subtitle lines.

https://translate.local:8443
https://translate.local:8443/api/health

Tune the server for latency or quality

Start with beam_size=1. If subtitles are fast but translation is rough, try beam_size=3. If the server is shared with other GPU work, step down to medium.

# Faster
$env:WHISPER_MODEL="large-v3"
$env:WHISPER_COMPUTE_TYPE="float16"
$env:WHISPER_BEAM_SIZE="1"
$env:SHOW_SOURCE_TEXT="true"

# More careful decoding
$env:WHISPER_MODEL="large-v3"
$env:WHISPER_COMPUTE_TYPE="float16"
$env:WHISPER_BEAM_SIZE="3"
$env:SHOW_SOURCE_TEXT="true"

# Lighter fallback
$env:WHISPER_MODEL="medium"
$env:WHISPER_COMPUTE_TYPE="int8_float16"
$env:WHISPER_BEAM_SIZE="1"
$env:SHOW_SOURCE_TEXT="true"

Prepare the server PC

Install current NVIDIA drivers, Python 3.11 or 3.12, Git, CUDA 12, and cuDNN 9. FFmpeg is optional for file conversion tests. Give the PC a stable LAN address such as 192.168.1.50.

Create the Python backend

Use the repo's requirements file so the backend matches the app code, including faster-whisper and WebRTC VAD. FastAPI's WebSocket guide explains the browser-to-server connection pattern.

python -m venv .venv
.\.venv\Scripts\Activate.ps1
pip install -r requirements.txt

Run the server on the LAN interface

Binding to 0.0.0.0 lets other devices on the network reach the backend. Uvicorn documents this through its --host setting.

$env:WHISPER_MODEL="large-v3"
$env:WHISPER_DEVICE="cuda"
$env:WHISPER_COMPUTE_TYPE="float16"
uvicorn live_translate.app:app --host 0.0.0.0 --port 8000

Add HTTPS for browser microphones

Browsers allow microphone capture only in secure contexts such as HTTPS or localhost. For phones and other laptops on the LAN, place Caddy, nginx, or another TLS reverse proxy in front of FastAPI and expose wss://. Caddy is a convenient first choice because it can handle local HTTPS.

https://translate.local
  -> FastAPI health:  http://127.0.0.1:8000/api/health
  -> FastAPI socket:  ws://127.0.0.1:8000/ws

Connect the web app

Open the GitHub Pages or local static frontend, enter the backend origin such as https://translate.local, then start the microphone.

Install the local runtime

Use the same Python backend, but start with a lighter model on laptop hardware.

python -m venv .venv
.\.venv\Scripts\Activate.ps1
pip install -r requirements.txt

Start in safe laptop mode

Use small first on an A1000-class laptop. Move to medium only after the safe preset is stable.

$env:WHISPER_MODEL="small"
$env:WHISPER_DEVICE="cuda"
$env:WHISPER_COMPUTE_TYPE="int8_float16"
$env:WHISPER_BEAM_SIZE="1"
python main.py

Open the browser UI

Because the app is on the same machine, localhost is a secure browser context for microphone access.

http://127.0.0.1:8000

Tune for speed or quality

Use base on CPU when CUDA fails. Use medium with WHISPER_BEAM_SIZE=1 when small is too rough and the GPU has headroom.

# Fast preset
$env:WHISPER_MODEL="small"
$env:WHISPER_COMPUTE_TYPE="int8_float16"
$env:WHISPER_BEAM_SIZE="1"

# Quality preset
$env:WHISPER_MODEL="medium"
$env:WHISPER_COMPUTE_TYPE="int8_float16"
$env:WHISPER_BEAM_SIZE="1"