DEV Community

FuriousOfNight
FuriousOfNight

Posted on

Building Furious Mirror — My Journey Learning Realtime Android Streaming Architecture

Building Furious Mirror — My Personal Realtime Android Mirroring Project Inspired by scrcpy

Hi everyone 👋

Over the last months I’ve been studying realtime multimedia pipelines, desktop rendering, low latency streaming and Android screen transmission systems.

During this journey I started building a personal project called Furious Mirror, heavily inspired by the amazing work behind scrcpy.

The goal was never to “replace” scrcpy or compete against it.
Actually, scrcpy was one of the biggest reasons I became interested in understanding how realtime streaming architectures work internally.

So I decided to challenge myself and build my own implementation focused on experimentation, learning and architecture exploration.


What Furious Mirror currently does

The project currently supports:

  • Android screen mirroring
  • Low latency streaming
  • USB connection
  • Experimental Wi-Fi/IP mode
  • FFmpeg decoding pipeline
  • SDL2 GPU rendering
  • Hybrid connection switching
  • Realtime rendering architecture
  • Input forwarding experiments

Tech Stack

The current pipeline is based on:

Android MediaCodec
↓
Raw H264 AnnexB stream
↓
FFmpeg decoding
↓
SDL2 GPU rendering
Enter fullscreen mode Exit fullscreen mode

Main technologies used:

  • Python
  • FFmpeg / PyAV
  • SDL2
  • ADB
  • MediaCodec
  • Multithreaded rendering pipeline

About AI-assisted development

I also want to be honest about something:

A large part of the project was developed using AI-assisted orchestration.

I’m still learning low-level multimedia engineering and advanced software architecture, so AI became part of my workflow for:

  • prototyping,
  • refactoring,
  • experimentation,
  • debugging ideas,
  • and accelerating iteration.

But I also learned that AI alone does not magically build stable realtime software.

A LOT of manual testing, architectural decisions, debugging and experimentation were necessary to make the project actually work in practice.

Especially when dealing with:

  • threading,
  • rendering pipelines,
  • synchronization,
  • buffering,
  • transport stability,
  • and realtime decoding.

Current state of the project

The project is still evolving and there are many things I want to improve:

  • better Wi-Fi stability
  • audio support improvements
  • bitrate adaptation
  • better reconnect logic
  • more optimized rendering paths
  • broader Android compatibility

Some devices work perfectly.
Some older Android devices still present challenges.

And honestly, this has been one of the most interesting learning experiences I’ve had so far.


Why I’m sharing this

Mostly to:

  • learn from more experienced developers,
  • receive feedback,
  • improve the architecture,
  • and connect with people interested in multimedia systems and realtime streaming.

If anyone has experience with:

  • FFmpeg,
  • SDL2,
  • Android MediaCodec,
  • low latency streaming,
  • transport pipelines,
  • or realtime rendering,

I would genuinely love to hear your thoughts.


Repository

GitHub Repository:

https://github.com/furiousofnightt/furious-mirror

Thanks for reading 🙌

Top comments (6)

Collapse
 
superfunicular profile image
Super Funicular

Solid breakdown — the MediaCodec → AnnexB → FFmpeg → SDL2 chain reads cleanly. A few things I learned building a similar realtime pipeline on the camera side (Background Camera RemoteStream — Camera2 → MediaRecorder → RTMP for YouTube, Ktor for LAN serve):

Wi-Fi transport stability is dominated by buffer-stall recovery, not raw bitrate. The biggest reliability jump came from a short jitter buffer (~120ms) on the receiver plus explicit MTU probing on the LAN side — random consumer routers fragment packets in weird ways and FFmpeg's default reorder window will mask the symptom for ~3s before stalling. Worth instrumenting if Wi-Fi mode feels "fine until it isn't."

Older-device compatibility is mostly OEM-modified Android — Xiaomi MIUI, Huawei EMUI, OnePlus OxygenOS aggressively kill background processes regardless of WAKELOCK + foreground service. MIUI in particular needs a manual "no battery restrictions" toggle the user has to set. Worth detecting OEM at runtime and surfacing a one-tap deep-link to the right Settings page rather than silently dying after a few minutes.

Audio sync — heads up for when you wire audio in: MediaCodec audio path has subtle PTS drift if the encoder restarts on a config change (rotation, codec swap, etc.). We had to anchor everything to a monotonic clock and resample on the receiver to avoid 50-150ms drift accumulating over a 30-min session.

Will star the repo. Curious — are you planning to keep the SDL2 GPU rendering as the only path, or do you see a hardware-decode-direct-to-window route for headless use cases?

(Camera-side counterpart for anyone here who wants the capture side rather than mirroring: play.google.com/store/apps/details?id=com.superfunicular.digicam — same OEM headaches, hard-won lessons.)

Collapse
 
furiousofnightt_ profile image
FuriousOfNight

Thank you for the detailed feedback and for sharing your experience.

Fortunately, the current implementation has been very stable in my real-world usage. I've tested both USB and Wi-Fi sessions for over 2 hours during live game streaming without crashes or major stalls.

The Wi-Fi mode was tested mostly on a 2.4 GHz network, so occasional FPS drops can happen during demanding games such as Honor of Kings, but the stream usually recovers almost immediately and remains playable.

Regarding future development, Furious Mirror was primarily built as a personal project to solve my own streaming and mirroring needs. After achieving the performance and stability goals I wanted, I don't currently plan major architectural changes.

That said, your notes about jitter buffers, OEM-specific Android behavior and future audio synchronization challenges were extremely valuable, and I'll definitely keep them in mind if the project evolves further.

At the moment, most of my development efforts are focused on a new project called FuriousCam. It is a desktop camera streaming solution currently approaching completion, with roughly 90% of the core development finished.

Just like Furious Mirror, it was created to solve a real-world need of my own, and the results so far have been very promising. I hope to share the project publicly soon and showcase the development journey and final results with the community.

Thanks again for taking the time to analyze the architecture and for the GitHub star.

Collapse
 
superfunicular profile image
Super Funicular

Honor of Kings on 2.4GHz for 2 hours without dropping the session is a serious stability bar — that kind of soak surfaces every codec quirk, so respect for keeping the stream playable through the FPS dips instead of breaking the connection outright.

FuriousCam intrigues me — is that an Android-phone-as-webcam-into-desktop pipeline, or a fully native desktop capture path? We've been working the other direction (Android camera → embedded Ktor web server, browser viewer on the LAN) and the desktop-receiver side is where everyone seems to bolt on OBS as a stopgap. Curious whether you're planning a native viewer app or going browser/WebRTC for the desktop side.

Looking forward to seeing it ship. If you want to peek at how the Android end of a phone-camera-streamer pipeline shakes out in production, our build is here: play.google.com/store/apps/details?id=com.superfunicular.digicam — happy to compare Camera2 quirks if useful.

Thread Thread
 
furiousofnightt_ profile image
FuriousOfNight

Thank you!

FuriousCam is actually my second major real-time multimedia project after Furious Mirror.

The main goal is to turn an Android device into a webcam source for OBS Studio, Streamlabs, Discord, Zoom and similar desktop applications, while keeping the same philosophy I used for Furious Mirror: low latency, stability and minimal user setup.

The current architecture uses an Android-side temporary server deployed through ADB (no permanently installed Android app), H.264 encoding, FFmpeg/PyAV decoding on Windows, and a native desktop application built with PySide6.

At the moment the project already supports USB and Wi-Fi operation, automatic USB-to-Wi-Fi fallback, camera hot-swapping, virtual camera output, floating preview windows, persistent settings and real-time performance statistics.

One of the design goals was to avoid requiring browsers or WebRTC for the primary workflow. I wanted a fully native desktop experience where the user simply connects the device and immediately gets a webcam source available inside OBS and other software.

Like Furious Mirror, FuriousCam started as a personal tool created to solve a real need in my own streaming workflow. After months of testing and refinement it has reached roughly 90% completion and is already stable enough for daily use.

Audio support is one of the few major features still planned for future versions. Right now the focus has been on delivering a clean and reliable video pipeline before expanding the feature set.

I really appreciate your interest and the technical feedback you've shared. It's always great to talk with someone who has spent time dealing with the realities of Android multimedia pipelines and Camera2 behavior in production environments.

I hope to publicly share FuriousCam very soon, just like I did with Furious Mirror.

Thread Thread
 
furiousofnightt_ profile image
FuriousOfNight

By the way, after our conversation I downloaded your app to take a look.

I really liked the interface and the amount of configuration options available. It's clear that a lot of thought went into making it flexible for different use cases.

I have to admit I got a little lost at first trying to understand the workflow 😄, especially because I wasn't sure if I should use it through YouTube, LAN streaming or another method. But I found the concept interesting and it gave me a better understanding of the approach you took.

I also followed your YouTube channel and plan to watch more of the videos when I have some free time. It's always interesting to see how other developers solve real-world multimedia and streaming challenges.

Thanks again for taking the time to share your experience and for the great feedback on my projects.

Collapse
 
furiousofnightt_ profile image
FuriousOfNight

leave your support for the project, I thank you in advance