Can Your Favorite App Keep Up? [Part 3] Video Cold Startup Testing at 2 Mbps Using VPN

Wherever you are — on a farm in rural India, in an airport lounge in São Paulo, Brazil, connected to free Wi-Fi capped at 2 Mbps during peak hours, in a multi-user household in New York, USA with heavy usage and low-bandwidth plans, or in a café in Berlin, Germany — your internet connection speed might not be ideal.

That’s why optimizing apps for low-bandwidth environments is crucial. But will all apps perform equally well under such conditions? Will their performance remain consistent when connected to servers worldwide? These are the questions we set out to answer when conducting a VPN experiment at TestDevLab’s Audio and Video Testing Lab.

This is the final article in our three-part series, where we tested the video cold-startup performance of apps like Pinterest, Snapchat, TikTok, Triller, X, and YouTube. Using NordVPN, we simulated user locations in Berlin (Germany), London (UK), Tokyo (Japan), Mumbai (India), São Paulo (Brazil), Dubai (UAE), Mexico City (Mexico), Jakarta (Indonesia), and New York (USA), with the network speed fixed at 2 Mbps.

During this research, we observed two key aspects: the overall video cold-startup performance across different apps and how performance varied depending on the user’s location.

Why 2 Mbps? Because it’s challenging — and that’s exactly the point.
This is the kind of speed users might encounter in rural areas, on congested networks, or when data-saving modes are enabled. It’s far from ideal, but still within the realm of real-world use. The goal was to understand how each app handles video playback under tight bandwidth constraints — how long it takes to load, when the playback begins, and how quality holds up across locations.VPNs like NordVPN played a crucial role in this testing. While often used for privacy or accessing geo-restricted content, VPNs are also a powerful tool for QA engineers simulating global user experiences. By routing traffic through different regions, we could observe how geography impacts app behavior, even when the connection speed is uniformly low.

What did we learn?
At 2 Mbps, the differences are stark. Some apps adapted quickly, delivering decent playback with minimal delay. Others struggled — buffering, stalling, or taking noticeably longer to start. Even though the speed was consistent, performance varied significantly based on server location and how well the app handled poor network conditions.

At this speed, users don’t expect high quality but still want videos to load promptly without frustrating delays. While the connection limits performance, a smooth and timely playback experience remains essential.

👉 For an overview of both 10 Mbps and 2 Mbps limitations, check out the first blog in our three-part series.

Preparing the test setup

1. Devices & environment

The following environments were created and devices used:

• Mobile device used: Samsung Galaxy S21, Android 14
• Device used: Apple MacBook Pro 13.3, macOS Sequoia 15.0.1
• Network conditions: Bitrate limited to 10 Mbps

Example: A user is using mobile data (4G) while sitting in a busy airport lounge. They open the YouTube app, which hasn’t been used in the last few days. The app cold-starts and loads the home feed within a couple of seconds. Videos autoplay smoothly in good quality, with only a brief delay.

• Network: NordVPN

2. Apps tested

We tested popular social media apps for short video forms:

• Pinterest, 13.5.0
• TikTok, version 38.7.3
• YouTube, version 20.06.36
• Snapchat, version 12.29.0.49
• Triller, version v55.2b54
• X, version versionName=10.80.1-release.0

3. Video uploaded 

To ensure a fair comparison across different apps, we created a new user profile and uploaded the same video to the applications. Video complexity plays a crucial role in evaluating performance, as different apps may handle high spatial complexity differently.

Video specifications

• Duration: 15 seconds
• Frame rate (FPS): 30
• Bitrate: 13,969 Kbps
• Resolution: 1080x1920
• Codec: H.264 (High) (avc1 / 0x31637661)
• Color format: YUV 4:2:0 (Progressive)

Video complexity

• High spatial complexity – Contains highly detailed scenes, intricate textures, and high contrast.
• Low temporal complexity – Features a static camera and background, with only the subject (a dancing girl) in motion.

Testing methodology

We conducted this experiment in the period between February 18 and March 26, 2025, with a total test count of 1,394. The precondition for the user scenario is that the app is set in a cold start-up state (clear cache and data). Below are the steps we took for each test scenario:

• User opens the app and logs in
• User clicks search and searches for the account
• User opens the account
• User clicks on the second short/video
• User watches the video for some time 

Recording length

• 15 seconds for 10 Mbps tests

Tools used for data processing and collection

• FFmpeg—a free and open-source software designed for processing video and audio files.
• Text detection for startup time measurement
• VMAF—full reference video metric; additionally used in-house built video processing argument to ignore color differences (darker overall for some of the apps)
• Ntrex—TestDevLab’s in-house built tool for processing network
• Tempero—TestDevLab’s in-house built tool for data collection
• Viqubox—TestDevLab’s in-house built tool for audio, video, and network processing
• Speed test app
• What is My IP Address app

Key performance metrics

The following performance metrics were gathered and analyzed:

• Startup time - The time it takes for the video to load after clicking on a video.
• Startup + buffer time - The time it takes for the video to load after clicking on a video link, plus the duration of buffering events during recording time.
• Out-of-scope tests - Metric used for poor latency, when video playback does not start within the recording time (either 15 or 30 seconds, depending on network limitation). Out-of-scope tests are excluded from the average data calculation as video playback has not started, and no performance data has been gathered.
• VMAF - This full reference video quality metric was developed by Netflix. For this particular experiment, an ignore color difference argument is used when processing video quality because most of the apps have a darker overlay at the top or bottom of the screen.

Results & findings

Let's look at the results from two perspectives and what the overall app performance is like at a 2 Mbps limitation for the video cold startup scenario, and then let's look from a geographical perspective if there are differences in results.

App performance at 2 Mbps limitation

We noticed a relative difference between the tested server locations for all apps. Before we deep dive into the analysis of how apps perform differently across different locations, let's observe their overall performance at this challenging speed limitation of 2 Mbps.

The graph below shows video cold startup performance at 2 Mbps. The apps are listed in alphabetical order.

When comparing the apps’ overall performance, we see that Pinterest leads in Android app optimization for low-bandwidth environments, delivering faster content access even under limited network conditions. Video quality is good and stable for this high spatial complexity type of video. When compared to other tested apps, Pinterest video quality is on par with competitors like YouTube, TikTok, and Snapchat, and higher than Triller and X.

When looking at challenging network constraints like a 2 Mbps limitation, an important metric to take into consideration in the analysis is out-of-scope tests that indicate firstly the inconsistency in application playback start and secondly the inability to start the playback in test time. The average results are calculated from the tests that are within the scope, from the tests where the video playback actually starts in 30 seconds.

You might be wondering, given all the performed tests, why do we not increase the testing time and wait longer for the video playback to start? When you tap on the video, would you, as a regular user, wait longer for it to start playing? Isn't 30 seconds long enough in our busy lifestyle?

Although TikTok shows the next fastest performance after Pinterest, and depending on location, starts video playback from 2.29 seconds to 8.45 seconds, in almost all locations, there are some out-of-scope tests. The out-of-scope percentage varies from 4% to 15%, showing very high inconsistency in app performance. Video quality is on par with competitors like Pinterest, YouTube, and Snapchat.

On the other hand, X is a little slower than TikTok, but there are no out-of-scope tests for X, indicating results consistency and good performance at the 2 Mbps limitation. On average, X starts video playback in 4.48 to 8.15 seconds, depending on location. From the results, we can see that X prioritizes the video playback start time and fluidity over video image quality and drops its video image quality the most. We also tested X at a 10 Mbps limitation, and from the results, we noticed that on average, X keeps its video quality at VMAF 61 at 10 Mbps, but drops from VMAF 43 to VMAF 53 at 2 Mbps. Besides that, we see that video image quality differs from location to location, adapting to the startup time.

Although YouTube manages to start video playback faster compared to X, we noticed that YouTube playback is less smooth compared to X, requiring extra buffering during the playback. In our experience testing YouTube, we discovered its great inconsistency in startup time results. All tests happened to start playback within testing time, and there are no out-of-scope tests for YouTube, but at the same time, there is great inconsistency between the results. YouTube may ensure great fluidity and quick start, but the next test might show high outliers at playback start or high buffering. YouTube has a slightly higher video quality compared to other apps.

Next faster is Triller, which starts video playback in 8.13 seconds to 23 seconds, but is struggling at 2 Mbps, having high buffering of an extra 12.6 to 15.92 seconds.  Overall performance is poor. The startup and buffer time is in range from 21 to 27 seconds, and video quality drops to VMAF 47. Besides high startup + buffer time, there are out-of-scope tests for VPN to Brazil, VPN to NY, and the regular network.

Snapchat shows totally different behaviour from other apps- although it starts the video playback the slowest from competitors, once the playback is started, the app requires no extra buffering. Overall, Snapchat has lower startup + buffer time compared to Triller. Snapchat keeps its video quality high, and it is on par with competitors like Pinterest, TikTok, and YouTube. 

👉 For a deeper dive into startup time and video quality variations across different regions at 10 Mbps, check out this blog post.

Location-to-location view

We noticed a relative difference in results when testing different locations. When analysing results, we should keep in mind that we tested NordVPN specifically, not all VPN providers. From the 2 Mbps limitation results, we found that there is no location where all apps would perform better compared to other locations. As VPN.com explains:

• Apps use different content delivery networks, servers (DNS), and backend servers. 
• Some apps might have stronger infrastructure or more localized edge servers in some regions,
• Apps may be more aggressively optimized for their key markets,
• Apps may use different low-bandwidth adaptation strategies,
• Apps may use region-specific DNS routing.

Pinterest video playback starts from just 0.17–0.20 seconds in locations like New York (USA), Jakarta (Indonesia), London (UK), and Berlin (Germany) to 0.26 sec to 0.36 seconds in Mexico, Dubai (the United Arab Emirates), Mumbai (India), and Tokyo (Japan). In São Paulo (Brazil), where network constraints are typically more pronounced, Pinterest still manages a startup time of 0.83 seconds, outperforming its competitors. Buffering during video playback was noticed in 3 locations—Pinterest video playback buffered for 0.78 seconds in Mexico, 0.61 seconds in São Paulo (Brazil), and 0.45 seconds in Tokyo (Japan). Video quality is consistent between locations.

Opposite Pinterest, where slower performance was noticed in Mexico City (Mexico) and São Paolo (Brazil), Snapchat manages to start video playback quicker in Mexico and São Paolo (Brazil) compared to New York (USA), London (UK), Berlin (Germany) and Dubai (the United Arab Emirates). 9% of VPN to New York (USA) tests, 10% of VPN to London (UK) tests, 8% of Dubai (the United Arab Emirates) tests and 4% of Tokyo (Japan) and Jakarta (Indonesia) tests were out of scope when testing Snapchat video cold startup scenario at 2 Mbps network condition.

The regular network tests - the tests that were performed without using VPN to any specific server location. When testing TikTok, 6% of these tests were out of the scope at 2 Mbps.  Second-highest result observed for VPN to New York tests, but TikTok buffered more for these tests, in total having the highest startup + buffer time from the tested scope.

Other locations show better results compared to the regular network and VPN to NY. The lowest average startup + buffer time was noticed for Mexico City (Mexico), London (UK), and Tokyo (Japan) tests. TikTok is banned in India; therefore excluded from testing in this location. Out of scope tests show high inconsistency between tests: 6% regular network, 5% Mexico, 4% São Paulo (Brazil), 15% Berlin (Germany), 6% Dubai (The United Arab Emirates), 15% Tokyo (Japan), and 13% Jakarta (Indonesia).

Triller has shown the slowest performance on all locations compared to other apps. The highest startup time was noticed for regular network tests that were conducted without using a VPN to specific server locations.

Regular network tests have not only higher startup time compared to other locations for Triller, but also extremely high out-of-scope test percentage—84% of tests were out of scope. When analysing the trace file for tests within the scope, we noticed that Triller connected to a server in Montreal (Canada). The test setup was located in Riga (Latvia), and no closer European servers were used. 

Around 4% of VPN to New York (USA) and 5% of São Paulo (Brazil) tests were out of scope for Triller.

We observed that for all other apps, test results from Jakarta (Indonesia) show either some of the lowest startup+ buffer times or fall somewhere in the middle compared to other locations. However, for X, the results are the opposite — Jakarta (Indonesia) shows the highest startup + buffer times. Next higher startup +buffer time for X noticed in São Paulo (Brazil). 

Fastest startup + buffering time for X noticed for tests using regular network, VPN to New York (USA), Berlin (Germany), and London (UK). Compared to other tested apps, X has less stable video quality across the tested locations.

On a 2 Mbps connection, X ensures the highest video quality (and also the lowest startup time) when testing a regular network without using a VPN. The next highest video quality was noticed for New York (USA), Tokyo(Japan), Mumbai (India), and Jakarta (Indonesia).

When analyzing startup time, YouTube performs the slowest in Brazil — video playback starts in an average of 11.19 seconds, compared to 4–6 seconds in other tested locations. But at the same time, YouTube requires less buffering in São Paulo (Brazil). Overall, the highest startup + buffer time was observed when using VPN servers in New York (USA), followed by Tokyo (Japan) and Dubai (UAE). In contrast, Jakarta (Indonesia) showed faster startup + buffer time. YouTube delivers the highest video playback quality at 2 Mbps among all tested apps.

Key findings

When comparing app behavior at 2 Mbps, we observed significant differences in performance and consistency.

• Pinterest stands out with the fastest and most reliable video startup times across all tested locations. It delivers smooth playback with minimal buffering and maintains high video quality, proving to be the best-optimized app for low-bandwidth conditions.
• TikTok ranks second in startup time, with results ranging between 2.29s and 8.45s, but it suffers from inconsistency, having out-of-scope tests for some of the locations.
• X has slower startup times than TikTok but has no out-of-scope tests. 
• YouTube shows fast startup but inconsistent playback quality and buffering. Although it avoids out-of-scope tests, results vary significantly between runs and locations. YouTube has slightly higher video quality than other platforms at this bandwidth.
• Triller shows overall poor performance, with high startup times and heavy buffering. The majority of regular network tests fell out of scope. 
• Snapchat shows a unique pattern — the slowest startup but no buffering, resulting in more stable overall playback than Triller. 

Location-to-location insights

From a geographic perspective, no single VPN server location universally outperformed others across all apps. This suggests the impact of factors such as:

• CDNs and backend infrastructure
• Region-specific optimization
• DNS routing strategies
• App-specific handling of low-bandwidth conditions

Specific highlights

• Pinterest consistently starts playback in under 0.4 seconds in locations like New York (USA), London (UK), Berlin (Germany), and Jakarta (Indonesia). Even in São Paulo (Brazil), it outperforms competitors with a startup time of just 0.83s.
• Snapchat performs better in Mexico City (Mexico) and São Paulo (Brazil) than in major locations like London or New York — an inverse trend compared to other apps.
• TikTok exhibits high variance in out-of-scope results, especially in Berlin (Germany), Tokyo (Japan), and Jakarta (Indonesia), showing that performance heavily depends on location.
• Triller underperforms across the board. In regular network tests, the app routed through Montreal (Canada) instead of a closer server, resulting in 84% out-of-scope tests.
• X performs faster on the regular network and Western servers (New York, London, Berlin). However, in Jakarta (Indonesia) and São Paulo (Brazil), it records the slowest startup + buffer times.
• YouTube struggles most in São Paulo (Brazil) with an average startup of 11.19s, while Jakarta (Indonesia) offers the smoothest experience. Despite inconsistencies, it delivers the highest video quality at 2 Mbps.

Below is an example of app performance using VPN to São Paulo (Brazil) in a 2 Mbps limitation test. When video playback starts, the video is tinted green.

Final thoughts

This VPN-based experiment reveals that app optimization and geographic server distribution play critical roles in video playback experience, especially under low-bandwidth (2 Mbps) conditions.

• Pinterest emerges as the top performer, with a fast, stable startup and consistent quality across regions.TikTok offers fast starts but suffers from stability issues, while X favors speed over quality.
• Snapchat and Triller followed contrasting patterns — Snapchat maintained high quality with slower starts, while Triller struggled across the board, especially with out-of-scope results and poor server routing.
• X prioritized fast startup and smooth playback, but at the cost of video quality, especially in low-bandwidth conditions.
• YouTube provided strong video quality but showed high variability in buffering and startup times depending on the location.

From a regional perspective, no location is universally optimal, but servers in Berlin (Germany), Jakarta (Indonesia), London (UK), and New York (USA) often performed better, though not consistently across all apps.

These findings emphasize that building global video platforms requires careful attention to infrastructure, content delivery strategy, and performance consistency, particularly in emerging markets with limited bandwidth.

Is your video app fast enough for your users? Contact us and learn more about our video testing services and how we can help optimize your platform's video performance.

Appendix

This appendix includes:

• Screenshots of speed tests, demonstrating network limitations in all tested locations.
• Screenshots of What is My IP Address, showing IP addresses and corresponding locations.

Speed tests

What is My IP Address