Can Your App Keep Up? [Part 2] Video Cold Startup Testing on 10 Mbps Using VPN

From checking notifications in Tokyo to catching up on trends in São Paulo, modern users are constantly on the move. Whether you’re commuting, traveling for business, or working remotely from a new city, short-form video has become the go-to for quick content. But how do these apps perform when the scenery and the server change?

At TestDevLab’s Audio and Video Testing Lab, we set out to answer a key question: Can leading short-form video apps maintain a fast, responsive experience at 10 Mbps, regardless of where in the world you’re watching from?

This is the second blog post in our three-part series, where we tested the video cold startup performance of apps like Pinterest, Snapchat, TikTok, Triller, X, and YouTube. Using NordVPN, we simulated user locations in Berlin (Germany), London (UK), Tokyo (Japan), Mumbai (India), São Paulo (Brazil), Dubai (UAE), Mexico City (Mexico), Jakarta (Indonesia), and New York (USA), while keeping the network speed fixed at 10 Mbps.

Why 10 Mbps? Because it’s realistic.
It’s the kind of bandwidth users typically experience in hotels, coffee shops, or while tethered to a mobile hotspot — not blazing fast, but still very usable. The goal was to observe how long each app takes to load and start playing a video right after launch, and whether location impacts performance even when the speed remains constant.

VPNs like NordVPN were instrumental in recreating real-world conditions. While they’re commonly associated with securing connections or bypassing geo-restrictions, VPNs are invaluable for QA teams and engineers testing performance from a global perspective. By tunneling through different servers, we could accurately mimic user sessions in multiple regions.

What did we learn?
Even with the same bandwidth, not all regions — or apps — perform equally. Some apps loaded almost instantly, while others lagged, making a significant difference in how smooth the user experience feels.

At 10 Mbps, users aren’t expecting perfection, but they’re still expecting playback without the wait.

👉 For an overview of both 10 Mbps and 2 Mbps limitations, check out the first blog in our three-part series.

Preparing the test setup

1. Devices & environment

The following environments were created and devices used:

• Mobile device used: Samsung Galaxy S21, Android 14
• Device used: Apple MacBook Pro 13.3, macOS Sequoia 15.0.1
• Network conditions: Bitrate limited to 10 Mbps

Example: A user is using mobile data (4G) while sitting in a busy airport lounge. They open the YouTube app, which hasn’t been used in the last few days. The app cold-starts and loads the home feed within a couple of seconds. Videos autoplay smoothly in good quality, with only a brief delay.

• Network: NordVPN

2. Apps tested

We tested a selection of popular apps that support short-form video content.

• Pinterest, 13.5.0
• TikTok, version 38.7.3
• YouTube, version 20.06.36
• Snapchat, version 12.29.0.49
• Triller, version v55.2b54
• X, version versionName=10.80.1-release.0

3. Video uploaded 

To ensure a fair comparison across different apps, we created a new user profile and uploaded the same video to the applications. Video complexity is crucial in evaluating performance, as different apps may handle high spatial complexity differently.

Video specifications

• Duration: 15 seconds
• Frame rate (FPS): 30
• Bitrate: 13,969 Kbps
• Resolution: 1080x1920
• Codec: H.264 (High) (avc1 / 0x31637661)
• Color format: YUV 4:2:0 (Progressive)

Video complexity

• High spatial complexity – Contains highly detailed scenes, intricate textures, and high contrast.
• Low temporal complexity – Features a static camera and background, with only the subject (a dancing girl) in motion.

Testing methodology

We conducted this experiment in the period between February 18 and March 26, 2025, with a total test count of 1,394. The precondition for the user scenario is that the app is set in a cold start-up state (clear cache and data). Below are the steps we took for each test scenario:

• User opens the app and logs in
• User clicks search and searches for the account
• User opens the account
• User clicks on the second short/video
• User watches the video for some time 

Recording length

• 15 seconds for 10 Mbps tests

Tools used for data processing and collection

• FFmpeg—a free and open-source software designed for processing video and audio files.
• Text detection for startup time measurement
• VMAF—full reference video metric; additionally used in-house built video processing argument to ignore color differences (darker overall for some of the apps)
• Ntrex—TestDevLab’s in-house built tool for processing network
• Tempero—TestDevLab’s in-house built tool for data collection
• Viqubox—TestDevLab’s in-house built tool for audio, video, and network processing
• Speed test app
• What is My IP Address app

Key performance metrics

The following performance metrics were gathered and analyzed:

• Startup + buffer time - The time it takes for the video to load after clicking on a video link, plus the duration of buffering events during recording time.
• Out-of-scope tests - Metric used for poor latency, when video playback does not start within the recording time (either 15 or 30 seconds, depending on network limitations). Out-of-scope tests are excluded from the average data calculation as video playback has not started, and no performance data has been gathered.
• VMAF - This full reference video quality metric was developed by Netflix. For this particular experiment, an ignore color difference argument is used when processing video quality because most of the apps have a darker overlay at the top or bottom of the screen.
• Startup time - The time it takes for the video to load after clicking on a video.

Comparing results between locations

Imagine you’re scrolling through your phone on a 4G connection while waiting for your coffee in a crowded city café. You tap on a short-form video app you haven’t opened in days. In this everyday scenario with moderate network speed, some apps stand out as more responsive and faster at launching video playback across all tested global regions. 

Pinterest, TikTok, YouTube, and X generally provide quick playback, though their speed can vary depending on the region you’re connecting from.

Snapchat and Triller, however, struggle to keep up, showing consistently slower startup times across all regions, even under a stable 10 Mbps connection. For users on the go, that delay could be the difference between a quick scroll and a frustrating wait.

The graph below shows video cold startup performance on 10 Mbps. The apps are listed in alphabetical order.

👉 For a deeper dive into out-of-scope tests, startup time, and video quality variations across different regions at 2 Mbps, check out this blog post.

Let’s take a look at our findings with the VPN set to different locations, and how they compare to the regular network conditions as our baseline.

Regular network

Pinterest stands out, featuring a very quick startup time of 0.19s, no buffering, and a solid VMAF score of 73. YouTube, while delivering the highest VMAF at 79, is the only app that exhibits buffering. TikTok and X show moderate delays (0.51s and 0.93s, respectively), but without any buffering. Snapchat and Triller have the slowest startup times (3.76s and 3.9s), with Triller also registering the lowest VMAF score of 44.

VPN to New York (USA)

Pinterest shows a startup time of just 0.13s, no buffering, and a solid VMAF of 73, reflecting consistently good video quality.TikTok performs well, with a moderate startup time of 0.57s, no added buffering, and a VMAF of 70. While slightly behind Pinterest in video quality and speed, it still offers a smooth viewing experience.YouTube has a noticeably longer startup time of 1.65s, but does not show additional buffering in this location. It compensates for the slower load with a high VMAF of 79, making it the best in terms of video quality. Snapchat and Triller are the slowest to start, with startup times of 4.01s and 3.6s, respectively. Neither shows extra buffering. Snapchat maintains decent visual quality with a VMAF of 71, while Triller trails in video quality with a VMAF of 45, the lowest among the apps. X shows moderate performance, with a startup time of 1.55s and no buffering. Its VMAF is 61, placing it between TikTok and Triller in terms of video quality.

VPN to Mexico City (Mexico)

Pinterest shows a startup time of 0.23s, no buffering, and a consistent VMAF of 73, ensuring quick load times and solid video quality.TikTok follows with a startup time of 1.75s and no added buffering. Its VMAF of 70 keeps it competitive, though both speed and quality are behind Pinterest.

YouTube starts at 1.94s and maintains the same value for startup + buffer time, indicating no buffering delays. It stands out with a high VMAF of 78, offering the best video quality among the tested apps in Mexico. Snapchat has a slower startup of 3.1s, with no buffering beyond that. Despite the delay, it offers a decent VMAF score of 71, placing it in the mid-range for quality. Triller is the slowest to start at 4s, with a slight buffering increase to 4.08s. It also ranks lowest in video quality, with a VMAF of 46, suggesting a poor visual experience overall. X shows a startup time of 2.1s with no additional buffering. Its VMAF of 61 is better than Triller’s but lags behind the other major platforms.

VPN to São Paulo (Brazil)

Pinterest's startup time is 0.16s, with no buffering, and a VMAF of 73, delivering consistent performance and user experience.TikTok starts significantly slower at 2.12s, but without additional buffering. Its VMAF of 70 is still within a good range, making it a decent option despite the delay.YouTube shows the slowest startup time at 7.15s, which may negatively impact user experience, although it maintains a high VMAF of 77, indicating excellent video quality once playback begins. Snapchat performs moderately with a startup time of 3.3s and no buffering. Its VMAF score of 71 suggests average video quality, balanced against the load time. Triller takes 4.07s to start and delivers the lowest video quality of all apps with a VMAF of 45, continuing its trend of underperformance across regions. X starts in 2.48s, faster than YouTube, Snapchat, and Triller, but slower than Pinterest and TikTok. Its VMAF score of 61 is modest, falling behind in video quality compared to top performers. 

Below is an example of app performance using VPN to São Paulo (Brazil) in a 10 Mbps limitation test. When video playback starts, the video is tinted green.

VPN to London (UK)

Pinterest shows a startup time of 0.23s, no buffering, and a VMAF score of 73, suggesting reliable and smooth performance.TikTok is a strong performer as well, with a quick startup time of 0.47s, no additional buffering, and a VMAF of 70, delivering smooth performance and decent video quality.YouTube starts slightly slower at 0.79s, again with no buffering. Its VMAF of 77 is the highest, making it ideal for users who prioritize video clarity.Snapchat shows slower responsiveness, with a startup of 3.24s, and a VMAF of 71—decent in terms of video quality but behind in speed.Triller improves a bit compared to other regions, loading in 2.66s with no buffering, but still lags in quality with a VMAF of 46, the lowest of the group.X takes 1.28s to start, with no buffering, and a VMAF of 61, putting it ahead of Triller but behind all other platforms in video fidelity.

VPN to Berlin (Germany)

Pinterest has a startup time of 0.21s, no buffering, and a VMAF score of 73, showing a balance of speed and video quality.TikTok follows with a startup time of 0.55s, also with no added buffering. Its VMAF of 70 is slightly lower than Pinterest’s, but still provides a solid user experience. YouTube loads in 0.8s, slightly slower than TikTok, with no buffering. It delivers a VMAF of 78, the highest in this region, making it ideal for high-quality video playback despite the marginal delay. Snapchat lags in speed with a startup of 3.12s, but it maintains a decent VMAF of 71, suggesting average video quality for a slower load time. Triller shows some improvement in speed here with a startup of 2.49s, but it continues to struggle in video quality with a VMAF of 45, the lowest among all apps tested. X starts in 1.21s, faster than Snapchat and Triller, but its VMAF of 61 places it in the mid-to-low range for video quality.

VPN to Dubai (The United Arab Emirates)

Pinterest has a startup time of 0.3s, no buffering, and a VMAF of 73, indicating both speed and high video quality. TikTok follows with a much higher startup time of 1.45s, no added buffering, and a VMAF of 70, maintaining a balanced performance. YouTube shows a longer startup time of 2.24s but compensates with strong video quality (VMAF 78), second only to Pinterest in overall experience. Snapchat loads in 3.18s and delivers a VMAF of 71, which is decent, but the relatively slower startup may affect initial engagement. Triller is one of the slowest apps to start at 4.03s and continues to score the lowest in video quality (VMAF 45), making it the weakest performer. X starts in 1.89s and provides moderate video quality (VMAF 61). It’s quicker than Snapchat and Triller but still lags behind Pinterest, TikTok, and YouTube.

VPN to Mumbai (India)

Pinterest shows a startup time of 0.2s, no buffering, and a VMAF of 73, indicating good video quality and a smooth user experience. YouTube experiences a slower startup time of 3.06s, with no buffering. Despite this, it has the highest VMAF of 77, indicating superior video quality. Snapchat has a startup time of 3.33s and shows no added buffering, with a VMAF of 71, offering decent video quality. Triller starts slow at 4.73s and has the lowest VMAF at 45, suggesting poor video quality despite its delay. X has a startup time of 2.91s, with no added buffering. Its VMAF of 62 is moderate compared to the other apps.

Note: TikTok is currently banned in India; therefore, no tests were conducted using a VPN connection to India for this app.

VPN to Tokyo (Japan) 

Pinterest delivers a startup of 0.26s, no additional buffering, and a high VMAF of 73, indicating consistently good video quality. TikTok follows with a startup time of 1.56s and a slightly increased buffering time (1.58s total), but maintains decent video quality with a VMAF of 70.YouTube has one of the longest startup times at 2.75s and shows additional buffering (total 2.85s). Surprisingly, its VMAF is only 72, lower than in other regions, where it typically leads. Snapchat starts slowly at 4.21s, with no added buffering, and offers moderate video quality (VMAF 71)—similar to TikTok but with a noticeably slower load time. Triller also performs poorly in Japan, with a startup time of 3.97s and the lowest video quality (VMAF 46), suggesting a less optimized experience. X loads in 2.56s, showing no buffering beyond startup, and maintains a VMAF of 61, making it a mid-tier performer in both speed and quality.

VPN to Jakarta (Indonesia)

Pinterest shows a startup time of 0.23s, no buffering, and a VMAF of 73, reflecting both speed and reliable video quality. TikTok starts up in 1.75s with no additional buffering and a VMAF of 70, offering a decent balance between responsiveness and video quality, though not quite at Pinterest’s level. YouTube shows a startup time of 1.94s, matching its startup + buffer time, meaning no extra delays. It again leads in video quality with the highest VMAF of 78, making it ideal for users prioritizing visual clarity. Snapchat loads more slowly at 3.1s, with no added buffering. Its VMAF of 71 puts it in the mid-range for video quality, slightly ahead of TikTok.Triller performs the worst, with the slowest startup at 4s, a minor increase to 4.08s including buffering, and a VMAF of just 46, suggesting the lowest video quality in this group. X has a startup time of 2.1s, no buffering, and a VMAF of 61, outperforming Triller in both speed and quality, but still lagging behind the other major platforms.

Key findings

Pinterest shows quick startup times across all regions, with no buffering and a VMAF score over 73, indicating a balance of speed and video quality. It shows faster performance in VPN tests to New York (USA) and São Paulo (Brazil), followed by quick starts in Mumbai (India), Berlin (Germany), London (UK), and Jakarta (Indonesia). Startup times are slightly slower but still strong in Tokyo (Japan) and Dubai (UAE).

Snapchat typically has slower startup times, ranging from 3.1s to 4.21s, but maintains a respectable VMAF score of 71, positioning it as a middle-of-the-pack performer. Video playback starts quicker in Mexico City (Mexico), Jakarta (Indonesia), Berlin (Germany), Dubai (UAE), and London (UK), but is slower in Mumbai (India), São Paulo (Brazil), Tokyo (Japan), and New York (USA).

TikTok shows moderate startup times (0.47s to 2.12s) and stable video quality, with a VMAF of around 70. While it performs well, it falls behind Pinterest in terms of speed and video clarity. TikTok video playback startup times vary significantly across regions. It starts as quickly as ~0.5 seconds in New York (USA), London (UK), and Berlin (Germany) but takes over three times longer in Dubai (UAE), Tokyo (Japan), Jakarta (Indonesia), Mexico City (Mexico), and the slowest in São Paulo (Brazil) at 2.12 seconds.

Triller shows the weakest overall performance among the tested apps. It starts faster in London (UK) and Berlin (Germany) but delivers its slowest performance in VPN tests to São Paulo (Brazil), Mumbai (India), and Jakarta (Indonesia).

With startup times ranging from 1.21s to 2.91s, X consistently lags behind Pinterest and YouTube in video quality (VMAF 61) but performs better than Snapchat and Triller. Similar to other apps, X starts playback faster in London (UK) and Berlin (Germany). The next fastest start times are observed in New York (USA), Dubai (UAE), and Jakarta (Indonesia). Slower startup times are recorded in São Paulo (Brazil), Tokyo (Japan), and Mumbai (India).

YouTube has longer startup times, ranging from 1.65s to 7.15s, but it also consistently achieves the highest video quality, with VMAF scores reaching up to 79. Video playback starts as quickly as 0.8 seconds in London (UK) and Berlin (Germany), but performance is less consistent in other regions. The next fastest locations are New York (USA), Jakarta (Indonesia), Mexico City (Mexico), Dubai (UAE), Tokyo (Japan), and Mumbai (India). The slowest and most inconsistent performance is observed in São Paulo (Brazil).

• VMAF score: YouTube frequently leads in video quality with scores up to 79, while Pinterest, Snapchat, and TikTok also deliver high video quality.
• Speed: On average, Pinterest has the fastest startup time at 0.22 seconds. TikTok and X also perform well, while Snapchat and Triller are noticeably slower. YouTube’s startup time varies significantly, from 0.79 seconds in the UK to 7.15 seconds in Brazil.

Conclusion

From our comparison of app startup times and video quality (VMAF), Pinterest showed consistent startup times across all regions, with an average of 0.19 seconds on regular networks and a stable ~0.2 seconds across multiple locations. Additionally, Pinterest maintained a high and steady VMAF score of 73, ensuring reliable visual quality across all regions.

YouTube ranked as the top performer in video quality, consistently achieving the highest VMAF scores (up to 79). However, its startup times were relatively slower, with significant delays in regions like Brazil (7.15s) and Japan (2.75s). Despite the slower start, YouTube’s high-quality video made it a preferred option for users prioritizing visual fidelity.

TikTok demonstrated competitive startup times, ranging from 0.47s to 2.12s. Its video quality remained stable across all regions, with a VMAF score of 70.

X delivered consistent, moderate startup times between 0.93s and 2.91s across regions, along with a relatively low but steady VMAF score of 61. While not the fastest or highest in video quality, X provided a dependable yet unremarkable performance.

Snapchat maintained a steady VMAF score of 71 but struggled with slower startup times, ranging from 3.1s to 4.21s. This slower performance, combined with lower video quality compared to Pinterest and YouTube, placed Snapchat lower in the rankings.

Triller had the lowest video quality, with startup times ranging from 2.49s to 4.73s and VMAF scores as low as 44. This combination of slower performance and poor video quality positioned Triller as the weakest performer in this comparison.

Is your video app fast enough for your users? Contact us and learn more about our video testing services and how we can help optimize your platform's video performance.

Key takeaways

• Most apps demonstrate faster startup times for VPN tests in Berlin (Germany), London (UK), and New York (USA). 
• Locations like Tokyo (Japan), Mumbai (India), São Paulo (Brazil), and Mexico City (Mexico) generally experience slower startup times. 
• Pinterest stands out by delivering a combination of fast startup times and high-quality visuals. 
• YouTube excels in video quality, but struggles with slower startup times and shows inconsistencies in overall performance.
• TikTok provides balanced performance, though some regional variations are noticeable. 
• X offers moderate, reliable performance without any standout features. 
• Snapchat and Triller lag in both speed and quality, with Snapchat exhibiting slower startup times and Triller showing both slow startup times and poor video quality.

Appendix

This appendix includes:

• Screenshots of speed tests, demonstrating network limitations in all tested locations.
• Screenshots of What is My IP Address, showing IP addresses and corresponding locations.

Speed tests at 10Mbps

What is My IP Address