Testing Android apps on real carrier networks and SIMs

This piece is about a small mobile team, or a solo developer, who has an Android app that passes everything in CI and still gets one-star reviews from people on 4G. The app works on office wifi, works on the emulator, and works on the dev’s own phone. Then a user on a prepaid SIM says login hangs, or the OTP never fills in, or the app thinks they are in the wrong country. I am writing this from the operator side, as someone who runs real phones in Singapore, so the examples lean that way. What follows is the setup I would build for that job, the failure modes I know come with it, and what it costs in effort.

The headline result is modest and useful. A real SIM test pass will not find every network bug, but it finds a specific class of bugs that nothing else in your pipeline can: the ones caused by the carrier sitting between your phone and your server. Once you have a repeatable checklist, it is a one-afternoon job per release candidate, not a project.

The setup

The minimum real-network rig is small.

• a physical Android phone with a working data SIM, ideally a phone you can leave plugged in
• a second SIM from a different carrier, because behaviour differs more between carriers than between phones
• a laptop with Android platform-tools installed (run adb version and note it in your test log)
• your debug or internal-test build, installed over USB or wifi adb
• a server or endpoint you control that echoes back what it sees: source IP, IP family, headers, and timing

That last item is the one people skip, and it is the most valuable. A tiny echo endpoint tells you what your backend actually receives from a carrier network. Without it you are guessing from the phone side only.

For the device side I use a short set of adb commands. These are standard and documented in the Android Debug Bridge docs:

adb devices
adb shell getprop gsm.operator.alpha
adb shell getprop gsm.sim.state
adb shell dumpsys telephony.registry
adb shell ip route get 1.1.1.1
adb shell ip -6 addr show
adb logcat -b radio

What they give you: gsm.operator.alpha is the registered carrier name, gsm.sim.state tells you if the SIM is READY, dumpsys telephony.registry shows the current data and service state, and the two ip commands show which route and which address families the phone is using. logcat -b radio is noisy but is where handover and registration messages land. Property names and dump formats can vary by Android version and vendor build, so run them once on your own device and keep the output as your baseline.

To toggle the data connection between test steps, adb shell svc data disable and adb shell svc data enable are the simplest. For airplane mode, newer builds expose adb shell cmd connectivity airplane-mode enable and disable. I believe that arrived around Android 11, but check with adb shell cmd connectivity on your build, and fall back to the quick settings tile by hand if it is missing.

Scale for this rig is one to four phones. Cost is the phones you already own, plus a prepaid data SIM per carrier; check current plans on the carrier sites. If you ever grow past a handful of devices, the host side starts to matter, and I wrote up the USB hub and power limits in host limits when running many android phones.

If you do not want to own the phones, there is the rented route. cloudf.one rents real Android phones in Singapore on dedicated hardware, each with a persistent Singapore mobile IP, controlled from the browser. Two honest limits: it is Singapore only, so it will not help if your users are elsewhere, and there is no self-service adb. adb and Appium reach a rented phone only through a WireGuard tunnel that we set up on request (email [email protected]). If your test is manual poking through the browser, that is enough. If you need scripted adb runs, plan for the tunnel setup step.

Only test apps and devices you own or are authorised to test. Everything below assumes your own builds.

What worked

Five tactics gave me the most signal for the least effort.

Run the echo endpoint check first

Before testing the app at all, open your echo endpoint from the phone’s browser on mobile data, then on wifi. Compare source IP, IP family and any forwarded headers. In practice this shows you three things in one minute: whether the phone is behind carrier-grade NAT, whether you are arriving over IPv6 or IPv4, and what country your geo-IP vendor thinks you are in.

Carrier-grade NAT means many subscribers share a public address. The shared address space is defined in RFC 6598 (100.64.0.0/10). If the phone’s local address sits in that range or a private one, and your echo endpoint sees a different public address, you are behind CGNAT. The practical consequence for your app: any logic that treats an IP as one user is wrong. Per-IP rate limits can lock out unrelated people on the same carrier address, and IP-based fraud rules can flag a whole neighbourhood.

Test IPv6-only and 464XLAT paths on purpose

Some mobile networks give the phone an IPv6-only connection and translate to IPv4 for legacy traffic, using the approach described in RFC 6877 (464XLAT). Android supports this through its CLAT component. Most apps survive it. The ones that break are apps with hardcoded IPv4 literals in config, old socket code that binds to 0.0.0.0, or SDKs that parse IPs with a regex that only knows dotted quads.

The test is simple. Search your code and config for IPv4 literals. Then run the app on a SIM where ip -6 addr show shows a global address and watch for failures. Whether a given Singapore carrier hands out IPv6, and in what mode, is something to check on your own SIM with the commands above.

Force a network handover in the middle of a request

Start a large upload or a long-polling call on wifi, walk out of range or turn wifi off, and see what the app does when traffic moves to cellular. Then do the reverse. Your TCP connection dies, your IP changes, and anything pinned to the old connection has to recover.

On the phone I do it with adb shell svc wifi disable mid-transfer, then svc wifi enable later. The things to watch in logcat are your HTTP client’s retry behaviour, whether resumable uploads actually resume, and whether websocket reconnect logic backs off or hammers the server. This is the bug class emulators are worst at, because the emulator’s network is a single virtual link that never really changes under you.

Use a real SIM for the SMS OTP flow

If your app verifies a phone number by SMS, you need a SIM that can receive. The thing to test is not only “does the code arrive” but “does the app pick it up”. Google documents the SMS Retriever API, which lets an app read a verification message without the broad SMS permission, provided the message ends with your app’s hash. Test three cases: message arrives while the app is foregrounded, message arrives while it is backgrounded, and message arrives late, after your own timeout.

Delivery is also a carrier matter. Messages from a short code or alphanumeric sender can be delayed or filtered differently per carrier, and international senders behave differently again. Keep a note of which sender type you used. Do not point this at services you are not authorised to test. Use your own verification flow.

Compare two carriers, not two phones

When a bug shows up, swap the SIM into the same phone before you swap phones. If the bug follows the SIM, it is the network. If it follows the phone, it is the device or OEM build. If you do have to separate device from OS behaviour, the notes in root vs non-root android automation cover which diagnostics you can get without modifying the phone.

What broke

The emulator told me everything was fine

The Android Emulator can shape bandwidth and latency. The emulator command line docs list -netspeed and -netdelay for exactly that, and the console accepts sms send to fake an incoming message. Useful for UI states like “slow network” and “OTP arrived”. But what it cannot cover is the stuff above: there is no carrier NAT, no real IPv6-only translation, no real handover between radios, no actual SMS routing through a carrier, and the geo-IP is whatever your dev machine’s connection looks like. A green emulator run only tells you your code handles a slow link politely.

The fix is just scoping. I keep emulator tests for UI and timeout logic, and I keep a short list of “real SIM only” checks that block a release.

Geo-IP disagreed with reality

A mobile IP can be mapped to the wrong city, or the wrong country, by a geo-IP database, especially after a carrier moves address blocks. Your app might block, redirect or localise based on that. It is cheap to check: hit your echo endpoint on each SIM, run the returned IP through the geo-IP vendor you actually use, and record the result per carrier and date.

The workaround in your product is to never make geo-IP the only signal. Let users correct their region, and avoid hard blocks on a single database lookup. If you need to see what a Singapore mobile user sees, that is where a Singapore-only real-device rental such as cloudf.one earns its place. For any other country, you need a SIM or device there.

The SIM went quiet mid-test

Prepaid SIMs expire, run out of data, or get a plan-level throttle. The app looks broken, but the phone is just rate-limited. Now the first line of every test run is a plain speed and connectivity check in a browser, logged with the time. If dumpsys telephony.registry shows the data state is not connected, I stop and top up before touching the app.

The numbers

I will not give ROI or CAC for a QA exercise, because there is no honest way to do it without invented data. Here is the arithmetic you can do yourself.

• hardware: phones you already own cost nothing extra. A second SIM is the one new cost, plus whatever data you burn. Check current prepaid plans on each carrier’s site; do not trust a price from an article.
• time to build the checklist: one afternoon to write the echo endpoint and the command cheat sheet, based on my experience with small setups. Your first run will be slower.
• time per release candidate: the pass has roughly eight to ten checks (echo endpoint, CGNAT, IPv6, handover up and down, OTP foreground, OTP background, OTP late, geo-IP, second carrier). Budget an hour or two by hand per carrier. That is an estimate, not a measured figure.
• scaling ratio: one phone per carrier you care about, so two to four phones covers most small teams. Past that, the cost is host management, not SIMs.

The value side is what you avoid: a bad release that reaches users on one carrier and shows up as support tickets and store reviews. The cost of that depends on your app.

If you want to automate part of the pass, the choice of driver matters. My comparison in maestro vs appium for android covers when each is the better fit. For a real-SIM pass, the automation only needs to drive the app. The network steps (data toggles, SIM swaps) stay manual or adb-scripted.

Lessons

• build the echo endpoint first. Seeing what your server receives from a carrier network answers half your questions before you open the app.
• treat an IP as shared. Under CGNAT, one address is many people, so audit per-IP limits and fraud rules.
• remove IPv4 literals from config and code, then test on an IPv6 SIM. It is a grep job plus one test run.
• test handover as its own scenario. Moving between wifi and cellular mid-request is where retries, resumable uploads and reconnect logic get exposed.
• swap the SIM before the phone when you debug. It separates network faults from device faults fast.
• keep emulator tests and real-SIM tests as two separate lists. The first catches UI and timeout bugs, the second catches carrier bugs. Neither replaces the other.

One more practical point on legality. SIM registration rules differ by country, and in Singapore prepaid SIMs are tied to identity registration. Register SIMs in your own name or your company’s, and do not buy or use SIMs registered under someone else. This is not legal advice, so check the current rules with the regulator or your carrier.

Would I do it again

Yes, and I would do it earlier. The first time I put a real SIM in a test phone and opened an echo page, I learned more about how my traffic looked to a server in five minutes than I had from weeks of wifi testing. The effort is low, the checklist is short, and the bugs it catches are exactly the ones users cannot describe in a review.

What I would not do is oversell it. A real SIM pass covers one carrier in one place at one time. It will not tell you how your app behaves for every user in every region, and carrier behaviour can change without notice. Treat results as dated observations: write down the carrier, the date, the Android version and the output of the commands.

If you want the wider set of how-tos and tools around this, the blog index has the rest of the Android automation notes.

Written by Xavier Fok

disclosure: this article may contain affiliate links. if you buy through them we may earn a commission at no extra cost to you. verdicts are independent of payouts. last reviewed by Xavier Fok on 2026-10-13.