A therapist and a patient are on a video call. The therapist moves a slider on the dashboard, and the moving dot on the patient's phone changes speed. Beside the call, a transcript builds up in English or French as they talk. All of that worked in the simulators. Getting it onto a real iPhone through the App Store was the part that nearly didn't happen.
The situation
CareAi is a mobile therapy platform built with Expo and React Native, aimed mainly at European users. Patients browse therapists, book appointments and join video consultations from the app, and therapists run the session from a web dashboard. Because it carries medical data for people in Europe, how that data was stored mattered from the first day, not as a clean-up task before launch.
This is a software case study, not infrastructure work, and I did it alone. My part was the real-time media integration, the transcription agent, the EMDR socket layer, the privacy controls, and the release fixes that got the app into both stores.
What a live session needed
A session had to do three things at once. It had to carry video, it had to produce a live transcript, and it had to let the therapist run EMDR on the patient's screen. EMDR (Eye Movement Desensitization and Reprocessing) guides a patient through specific visual and audio patterns to help them process trauma, and it had to be fully interactive from the therapist's side.
I split that into two channels that run side by side in the same session. Agora carries the media and the transcription. Socket.IO carries the EMDR state.
Agora for video plus STT
Building the real-time media layer from scratch
Why: Agora gave me reliable video infrastructure and speech-to-text out of the box. That let me spend the effort on the transcription agent and session logic instead of media plumbing.
For EMDR, the app already used sockets for its real-time features, so I built on them rather than adding another transport. On the dashboard, a slider and buttons control the dot's direction, color and speed, the sound and the background. Each change goes out as a small JSON payload, and the patient's app applies the new state as soon as it arrives. That's what lets the therapist show, hide or change a pattern mid-session, almost as if they were in the same room.
The transcript needed more work. Stable video wasn't enough on its own; the requirement also included live speech-to-text in English and French. I built an Agora agent that joins the call, captures the live audio, stores the transcript in S3 and streams it to the dashboard as it's produced. It took some trial and error before it ran smoothly.
On top of the transcript I added an AI assistant that uses context from previous sessions and the live transcript to summarize key points and flag possible concerns. It was never meant to replace the therapist's judgement. The point was less manual note-taking and less context lost between sessions.
Designing the data handling in early
Two controls went in before most of the features: encryption at rest and in transit, and aliases in place of patient identifiers, so an exposed internal record doesn't reveal who the patient is. Putting them in early slowed the first stretch of work, mostly because aliasing limited how transcripts and chat could be stored and joined later. I took that cost rather than bolt the controls on later, once the data model had already grown around real identifiers.
What I'm not claiming is that the product is GDPR compliant. I implemented specific technical controls. Compliance is a legal decision that involves data-processing agreements, retention policy, lawful basis and erasure handling, and none of that is an engineering sign-off.
The build that undid its own fix
Everything worked in the simulators. The App Store build didn't: it failed on AgoraRTC bitcode errors on iOS.
The standard answer, which I found in forum threads, is a script that strips bitcode from the framework. I applied it and the build still failed. Debugging it on an M1 Mac, I found out why. The script was changing the local ios/ folder, but Expo generates that folder itself and ignores the local copy during the real build. The fix was correct. It was being applied somewhere that got thrown away every time.
So I moved the fix into Expo's own build lifecycle: a post-build script that runs after pod install and strips the bitcode from the AgoraRTC framework automatically. With that in place, the build went through.
A post-build Expo script that strips AgoraRTC bitcode after pod install
Patching the local ios/ folder, which Expo regenerates and ignores
Why: Expo owns native project generation, so any manual edit to ios/ is thrown away at build time. To survive, the fix has to run inside Expo's own build lifecycle.
This kind of bug hides between tools, and on this project most of the delivery risk sat there rather than in the feature code.
The result
The app went to both the App Store and the Play Store for client testing, with video consultation, live transcription, therapist-controlled EMDR and AI-assisted summaries working in the same session.
What's still open
The app shipped for client testing. If I owned it longer-term, these are the first things I'd instrument or build:
- Agora call quality. Call-drop and reconnect rates, because a dropped call interrupts an active therapy session.
- Transcription latency and accuracy per language. The English and French transcripts feed both the therapist's view and the AI summaries.
- EMDR state drift. Whether the patient's EMDR state ever drifts from what the dashboard sent, and how fast it recovers.
- Access auditing on stored transcripts. Who and what touches them in S3, given the medical data involved.
- Data lifecycle. Retention windows and erasure requests across transcripts, chat and session records. This wasn't built, and it's the gap between the controls that exist and any real compliance position.